1 <?xml version="1.0" encoding="iso-8859-1"?>
2 <!DOCTYPE chapter PUBLIC "-//Samba-Team//DTD DocBook V4.2-Based Variant V1.0//EN" "http://www.samba.org/samba/DTD/samba-doc">
6 <firstname>Alexander</firstname><surname>Bokovoy</surname>
8 <address><email>ab@samba.org</email></address>
12 <firstname>Stefan</firstname><surname>Metzmacher</surname>
14 <address><email>metze@samba.org</email></address>
17 <pubdate> 27 May 2003 </pubdate>
20 <title>VFS Modules</title>
23 <title>The Samba (Posix) VFS layer</title>
25 <para>While most of Samba deployments are done using POSIX-compatible operating systems,
26 there is clearly more to a file system than what is required by POSIX when it comes to
27 adopting semantics of NT file system. Since Samba 2.2 all file-system related operations
28 go through an abstraction layer for virtual file system (VFS) that is modelled after
29 both POSIX and additional functions needed to transform NTFS semantics.
33 This abstraction layer now provides more features than a regular POSIX file system could
34 fill in. It is not required that all of them should be implemented by your
35 particular file system. However, when those features are available, Samba would advertize them to a
36 CIFS client and they might be used by an application and in case of Windows client that
37 might mean a client expects even more additional functionality when it encounters
38 those features. There is a practical reason to allow handling of this snowfall without modifying
39 the Samba core and it is fulfilled by providing an infrastructure to dynamically load
40 VFS modules at run time.
43 <para>Each VFS module could implement a number of VFS operations. The way it does it is
44 irrelevant, only two things actually matter: whether specific implementation wants to cooperate
45 with other modules' implementations or not, and whether module needs to store additional
46 information that is specific to a context it is operating in. Multiple VFS modules could
47 be loaded at the same time and it is even possible to load several instances of the same
48 VFS module with different parameters.
52 <title>The general interface</title>
55 Each VFS operation has a vfs_op_type, a function pointer and a handle pointer in the
56 struct vfs_ops and tree macros to make it easier to call the operations.
57 (Take a look at <filename>include/vfs.h</filename> and <filename>include/vfs_macros.h</filename>.)
60 <para><programlisting>
61 typedef enum _vfs_op_type {
79 </programlisting></para>
81 <para>This struct contains the function and handle pointers for all operations.<programlisting>
83 struct vfs_fn_pointers {
88 int (*open)(struct vfs_handle_struct *handle,
89 struct connection_struct *conn,
90 const char *fname, int flags, mode_t mode);
91 int (*close)(struct vfs_handle_struct *handle,
92 struct files_struct *fsp, int fd);
93 ssize_t (*read)(struct vfs_handle_struct *handle,
94 struct files_struct *fsp, int fd, void *data, size_t n);
95 ssize_t (*write)(struct vfs_handle_struct *handle,
96 struct files_struct *fsp, int fd,
97 const void *data, size_t n);
98 SMB_OFF_T (*lseek)(struct vfs_handle_struct *handle,
99 struct files_struct *fsp, int fd,
100 SMB_OFF_T offset, int whence);
101 ssize_t (*sendfile)(struct vfs_handle_struct *handle,
102 int tofd, files_struct *fsp, int fromfd,
103 const DATA_BLOB *header, SMB_OFF_T offset, size_t count);
108 struct vfs_handles_pointers {
111 /* File operations */
113 struct vfs_handle_struct *open;
114 struct vfs_handle_struct *close;
115 struct vfs_handle_struct *read;
116 struct vfs_handle_struct *write;
117 struct vfs_handle_struct *lseek;
118 struct vfs_handle_struct *sendfile;
123 </programlisting></para>
126 This macros SHOULD be used to call any vfs operation.
127 DO NOT ACCESS conn->vfs.ops.* directly !!!
131 /* File operations */
132 #define SMB_VFS_OPEN(conn, fname, flags, mode) \
133 ((conn)->vfs.ops.open((conn)->vfs.handles.open,\
134 (conn), (fname), (flags), (mode)))
135 #define SMB_VFS_CLOSE(fsp, fd) \
136 ((fsp)->conn->vfs.ops.close(\
137 (fsp)->conn->vfs.handles.close, (fsp), (fd)))
138 #define SMB_VFS_READ(fsp, fd, data, n) \
139 ((fsp)->conn->vfs.ops.read(\
140 (fsp)->conn->vfs.handles.read,\
141 (fsp), (fd), (data), (n)))
142 #define SMB_VFS_WRITE(fsp, fd, data, n) \
143 ((fsp)->conn->vfs.ops.write(\
144 (fsp)->conn->vfs.handles.write,\
145 (fsp), (fd), (data), (n)))
146 #define SMB_VFS_LSEEK(fsp, fd, offset, whence) \
147 ((fsp)->conn->vfs.ops.lseek(\
148 (fsp)->conn->vfs.handles.lseek,\
149 (fsp), (fd), (offset), (whence)))
150 #define SMB_VFS_SENDFILE(tofd, fsp, fromfd, header, offset, count) \
151 ((fsp)->conn->vfs.ops.sendfile(\
152 (fsp)->conn->vfs.handles.sendfile,\
153 (tofd), (fsp), (fromfd), (header), (offset), (count)))
156 </programlisting></para>
161 <title>Possible VFS operation layers</title>
164 These values are used by the VFS subsystem when building the conn->vfs
165 and conn->vfs_opaque structs for a connection with multiple VFS modules.
166 Internally, Samba differentiates only opaque and transparent layers at this process.
167 Other types are used for providing better diagnosing facilities.
171 Most modules will provide transparent layers. Opaque layer is for modules
172 which implement actual file system calls (like DB-based VFS). For example,
173 default POSIX VFS which is built in into Samba is an opaque VFS module.
177 Other layer types (logger, splitter, scanner) were designed to provide different
178 degree of transparency and for diagnosing VFS module behaviour.
182 Each module can implement several layers at the same time provided that only
183 one layer is used per each operation.
186 <para><programlisting>
187 typedef enum _vfs_op_layer {
188 SMB_VFS_LAYER_NOOP = -1, /* - For using in VFS module to indicate end of array */
189 /* of operations description */
190 SMB_VFS_LAYER_OPAQUE = 0, /* - Final level, does not call anything beyond itself */
191 SMB_VFS_LAYER_TRANSPARENT, /* - Normal operation, calls underlying layer after */
192 /* possibly changing passed data */
193 SMB_VFS_LAYER_LOGGER, /* - Logs data, calls underlying layer, logging may not */
195 SMB_VFS_LAYER_SPLITTER, /* - Splits operation, calls underlying layer _and_ own facility, */
196 /* then combines result */
197 SMB_VFS_LAYER_SCANNER /* - Checks data and possibly initiates additional */
198 /* file activity like logging to files _inside_ samba VFS */
200 </programlisting></para>
207 <title>The Interaction between the Samba VFS subsystem and the modules</title>
210 <title>Initialization and registration</title>
213 As each Samba module a VFS module should have a
214 <programlisting>NTSTATUS vfs_example_init(void);</programlisting> function if it's staticly linked to samba or
215 <programlisting>NTSTATUS init_module(void);</programlisting> function if it's a shared module.
219 This should be the only non static function inside the module.
220 Global variables should also be static!
224 The module should register its functions via the
226 NTSTATUS smb_register_vfs(int version, const char *name, vfs_op_tuple *vfs_op_tuples);
227 </programlisting> function.
232 <varlistentry><term>version</term>
233 <listitem><para>should be filled with SMB_VFS_INTERFACE_VERSION</para></listitem>
236 <varlistentry><term>name</term>
237 <listitem><para>this is the name witch can be listed in the
238 <command>vfs objects</command> parameter to use this module.</para></listitem>
241 <varlistentry><term>vfs_op_tuples</term>
243 this is an array of vfs_op_tuple's.
244 (vfs_op_tuples is descripted in details below.)
251 For each operation the module wants to provide it has a entry in the
256 typedef struct _vfs_op_tuple {
265 <varlistentry><term>op</term>
266 <listitem><para>the function pointer to the specified function.</para></listitem>
269 <varlistentry><term>type</term>
270 <listitem><para>the vfs_op_type of the function to specified witch operation the function provides.</para></listitem>
273 <varlistentry><term>layer</term>
274 <listitem><para>the vfs_op_layer in whitch the function operates.</para></listitem>
279 <para>A simple example:</para>
282 static vfs_op_tuple example_op_tuples[] = {
283 {SMB_VFS_OP(example_connect), SMB_VFS_OP_CONNECT, SMB_VFS_LAYER_TRANSPARENT},
284 {SMB_VFS_OP(example_disconnect), SMB_VFS_OP_DISCONNECT, SMB_VFS_LAYER_TRANSPARENT},
286 {SMB_VFS_OP(example_rename), SMB_VFS_OP_RENAME, SMB_VFS_LAYER_OPAQUE},
288 /* This indicates the end of the array */
289 {SMB_VFS_OP(NULL), SMB_VFS_OP_NOOP, SMB_VFS_LAYER_NOOP}
292 NTSTATUS init_module(void)
294 return smb_register_vfs(SMB_VFS_INTERFACE_VERSION, "example", example_op_tuples);
301 <title>How the Modules handle per connection data</title>
303 <para>Each VFS function has as first parameter a pointer to the modules vfs_handle_struct.
307 typedef struct vfs_handle_struct {
308 struct vfs_handle_struct *next, *prev;
310 struct vfs_ops vfs_next;
311 struct connection_struct *conn;
313 void (*free_data)(void **data);
319 <varlistentry><term>param</term>
320 <listitem><para>this is the module parameter specified in the <command>vfs objects</command> parameter.</para>
321 <para>e.g. for 'vfs objects = example:test' param would be "test".</para></listitem>
324 <varlistentry><term>vfs_next</term>
325 <listitem><para>This vfs_ops struct contains the information for calling the next module operations.
326 Use the SMB_VFS_NEXT_* macros to call a next module operations and
327 don't access handle->vfs_next.ops.* directly!</para></listitem>
330 <varlistentry><term>conn</term>
331 <listitem><para>This is a pointer back to the connection_struct to witch the handle belongs.</para></listitem>
334 <varlistentry><term>data</term>
335 <listitem><para>This is a pointer for holding module private data.
336 You can alloc data with connection life time on the handle->conn->mem_ctx TALLOC_CTX.
337 But you can also manage the memory allocation yourself.</para></listitem>
340 <varlistentry><term>free_data</term>
341 <listitem><para>This is a function pointer to a function that free's the module private data.
342 If you talloc your private data on the TALLOC_CTX handle->conn->mem_ctx,
343 you can set this function pointer to NULL.</para></listitem>
348 <para>Some useful MACROS for handle private data.
352 #define SMB_VFS_HANDLE_GET_DATA(handle, datap, type, ret) { \
353 if (!(handle)||((datap=(type *)(handle)->data)==NULL)) { \
354 DEBUG(0,("%s() failed to get vfs_handle->data!\n",FUNCTION_MACRO)); \
359 #define SMB_VFS_HANDLE_SET_DATA(handle, datap, free_fn, type, ret) { \
361 DEBUG(0,("%s() failed to set handle->data!\n",FUNCTION_MACRO)); \
364 if ((handle)->free_data) { \
365 (handle)->free_data(&(handle)->data); \
367 (handle)->data = (void *)datap; \
368 (handle)->free_data = free_fn; \
372 #define SMB_VFS_HANDLE_FREE_DATA(handle) { \
373 if ((handle) && (handle)->free_data) { \
374 (handle)->free_data(&(handle)->data); \
379 <para>How SMB_VFS_LAYER_TRANSPARENT functions can call the SMB_VFS_LAYER_OPAQUE functions.</para>
381 <para>The easiest way to do this is to use the SMB_VFS_OPAQUE_* macros.
386 /* File operations */
387 #define SMB_VFS_OPAQUE_OPEN(conn, fname, flags, mode) \
388 ((conn)->vfs_opaque.ops.open(\
389 (conn)->vfs_opaque.handles.open,\
390 (conn), (fname), (flags), (mode)))
391 #define SMB_VFS_OPAQUE_CLOSE(fsp, fd) \
392 ((fsp)->conn->vfs_opaque.ops.close(\
393 (fsp)->conn->vfs_opaque.handles.close,\
395 #define SMB_VFS_OPAQUE_READ(fsp, fd, data, n) \
396 ((fsp)->conn->vfs_opaque.ops.read(\
397 (fsp)->conn->vfs_opaque.handles.read,\
398 (fsp), (fd), (data), (n)))
399 #define SMB_VFS_OPAQUE_WRITE(fsp, fd, data, n) \
400 ((fsp)->conn->vfs_opaque.ops.write(\
401 (fsp)->conn->vfs_opaque.handles.write,\
402 (fsp), (fd), (data), (n)))
403 #define SMB_VFS_OPAQUE_LSEEK(fsp, fd, offset, whence) \
404 ((fsp)->conn->vfs_opaque.ops.lseek(\
405 (fsp)->conn->vfs_opaque.handles.lseek,\
406 (fsp), (fd), (offset), (whence)))
407 #define SMB_VFS_OPAQUE_SENDFILE(tofd, fsp, fromfd, header, offset, count) \
408 ((fsp)->conn->vfs_opaque.ops.sendfile(\
409 (fsp)->conn->vfs_opaque.handles.sendfile,\
410 (tofd), (fsp), (fromfd), (header), (offset), (count)))
414 <para>How SMB_VFS_LAYER_TRANSPARENT functions can call the next modules functions.</para>
416 <para>The easiest way to do this is to use the SMB_VFS_NEXT_* macros.
421 /* File operations */
422 #define SMB_VFS_NEXT_OPEN(handle, conn, fname, flags, mode) \
423 ((handle)->vfs_next.ops.open(\
424 (handle)->vfs_next.handles.open,\
425 (conn), (fname), (flags), (mode)))
426 #define SMB_VFS_NEXT_CLOSE(handle, fsp, fd) \
427 ((handle)->vfs_next.ops.close(\
428 (handle)->vfs_next.handles.close,\
430 #define SMB_VFS_NEXT_READ(handle, fsp, fd, data, n) \
431 ((handle)->vfs_next.ops.read(\
432 (handle)->vfs_next.handles.read,\
433 (fsp), (fd), (data), (n)))
434 #define SMB_VFS_NEXT_WRITE(handle, fsp, fd, data, n) \
435 ((handle)->vfs_next.ops.write(\
436 (handle)->vfs_next.handles.write,\
437 (fsp), (fd), (data), (n)))
438 #define SMB_VFS_NEXT_LSEEK(handle, fsp, fd, offset, whence) \
439 ((handle)->vfs_next.ops.lseek(\
440 (handle)->vfs_next.handles.lseek,\
441 (fsp), (fd), (offset), (whence)))
442 #define SMB_VFS_NEXT_SENDFILE(handle, tofd, fsp, fromfd, header, offset, count) \
443 ((handle)->vfs_next.ops.sendfile(\
444 (handle)->vfs_next.handles.sendfile,\
445 (tofd), (fsp), (fromfd), (header), (offset), (count)))
454 <title>Upgrading to the New VFS Interface</title>
457 <title>Upgrading from 2.2.* and 3.0aplha modules</title>
461 Add "vfs_handle_struct *handle, " as first parameter to all vfs operation functions.
462 e.g. example_connect(connection_struct *conn, const char *service, const char *user);
463 -> example_connect(vfs_handle_struct *handle, connection_struct *conn, const char *service, const char *user);
467 Replace "default_vfs_ops." with "smb_vfs_next_".
468 e.g. default_vfs_ops.connect(conn, service, user);
469 -> smb_vfs_next_connect(conn, service, user);
473 Uppercase all "smb_vfs_next_*" functions.
474 e.g. smb_vfs_next_connect(conn, service, user);
475 -> SMB_VFS_NEXT_CONNECT(conn, service, user);
479 Add "handle, " as first parameter to all SMB_VFS_NEXT_*() calls.
480 e.g. SMB_VFS_NEXT_CONNECT(conn, service, user);
481 -> SMB_VFS_NEXT_CONNECT(handle, conn, service, user);
485 (Only for 2.2.* modules)
486 Convert the old struct vfs_ops example_ops to
487 a vfs_op_tuple example_op_tuples[] array.
490 struct vfs_ops example_ops = {
491 /* Disk operations */
492 example_connect, /* connect */
493 example_disconnect, /* disconnect */
495 /* Directory operations */
501 /* File operations */
510 example_stat, /* stat */
511 example_fstat, /* fstat */
512 example_lstat, /* lstat */
521 NULL, /* ftruncate */
528 NULL, /* fget_nt_acl */
529 NULL, /* get_nt_acl */
530 NULL, /* fset_nt_acl */
531 NULL, /* set_nt_acl */
533 NULL, /* chmod_acl */
534 NULL, /* fchmod_acl */
536 NULL, /* sys_acl_get_entry */
537 NULL, /* sys_acl_get_tag_type */
538 NULL, /* sys_acl_get_permset */
539 NULL, /* sys_acl_get_qualifier */
540 NULL, /* sys_acl_get_file */
541 NULL, /* sys_acl_get_fd */
542 NULL, /* sys_acl_clear_perms */
543 NULL, /* sys_acl_add_perm */
544 NULL, /* sys_acl_to_text */
545 NULL, /* sys_acl_init */
546 NULL, /* sys_acl_create_entry */
547 NULL, /* sys_acl_set_tag_type */
548 NULL, /* sys_acl_set_qualifier */
549 NULL, /* sys_acl_set_permset */
550 NULL, /* sys_acl_valid */
551 NULL, /* sys_acl_set_file */
552 NULL, /* sys_acl_set_fd */
553 NULL, /* sys_acl_delete_def_file */
554 NULL, /* sys_acl_get_perm */
555 NULL, /* sys_acl_free_text */
556 NULL, /* sys_acl_free_acl */
557 NULL /* sys_acl_free_qualifier */
562 static vfs_op_tuple example_op_tuples[] = {
563 {SMB_VFS_OP(example_connect), SMB_VFS_OP_CONNECT, SMB_VFS_LAYER_TRANSPARENT},
564 {SMB_VFS_OP(example_disconnect), SMB_VFS_OP_DISCONNECT, SMB_VFS_LAYER_TRANSPARENT},
566 {SMB_VFS_OP(example_fstat), SMB_VFS_OP_FSTAT, SMB_VFS_LAYER_TRANSPARENT},
567 {SMB_VFS_OP(example_stat), SMB_VFS_OP_STAT, SMB_VFS_LAYER_TRANSPARENT},
568 {SMB_VFS_OP(example_lstat), SMB_VFS_OP_LSTAT, SMB_VFS_LAYER_TRANSPARENT},
570 {SMB_VFS_OP(NULL), SMB_VFS_OP_NOOP, SMB_VFS_LAYER_NOOP}
576 Move the example_op_tuples[] array to the end of the file.
580 Add the init_module() function at the end of the file.
583 NTSTATUS init_module(void)
585 return smb_register_vfs(SMB_VFS_INTERFACE_VERSION,"example",example_op_tuples);
591 Check if your vfs_init() function does more then just prepare the vfs_ops structs or
592 remember the struct smb_vfs_handle_struct.
594 <member>If NOT you can remove the vfs_init() function.</member>
595 <member>If YES decide if you want to move the code to the example_connect() operation or to the init_module(). And then remove vfs_init().
596 e.g. a debug class registration should go into init_module() and the allocation of private data should go to example_connect().</member>
601 (Only for 3.0alpha* modules)
602 Check if your vfs_done() function contains needed code.
604 <member>If NOT you can remove the vfs_done() function.</member>
605 <member>If YES decide if you can move the code to the example_disconnect() operation. Otherwise register a SMB_EXIT_EVENT with smb_register_exit_event(); (Described in the <link linkend="modules">modules section</link>) And then remove vfs_done(). e.g. the freeing of private data should go to example_disconnect().
611 Check if you have any global variables left.
612 Decide if it wouldn't be better to have this data on a connection basis.
614 <member>If NOT leave them as they are. (e.g. this could be the variable for the private debug class.)</member>
615 <member>If YES pack all this data into a struct. You can use handle->data to point to such a struct on a per connection basis.</member>
618 e.g. if you have such a struct:
620 struct example_privates {
625 first way of doing it:
627 static int example_connect(vfs_handle_struct *handle,
628 connection_struct *conn, const char *service,
631 struct example_privates *data = NULL;
633 /* alloc our private data */
634 data = (struct example_privates *)talloc_zero(conn->mem_ctx, sizeof(struct example_privates));
636 DEBUG(0,("talloc_zero() failed\n"));
640 /* init out private data */
641 data->some_string = talloc_strdup(conn->mem_ctx,"test");
642 if (!data->some_string) {
643 DEBUG(0,("talloc_strdup() failed\n"));
647 data->db_connection = open_db_conn();
649 /* and now store the private data pointer in handle->data
650 * we don't need to specify a free_function here because
651 * we use the connection TALLOC context.
652 * (return -1 if something failed.)
654 VFS_HANDLE_SET_DATA(handle, data, NULL, struct example_privates, return -1);
656 return SMB_VFS_NEXT_CONNECT(handle,conn,service,user);
659 static int example_close(vfs_handle_struct *handle, files_struct *fsp, int fd)
661 struct example_privates *data = NULL;
663 /* get the pointer to our private data
664 * return -1 if something failed
666 SMB_VFS_HANDLE_GET_DATA(handle, data, struct example_privates, return -1);
668 /* do something here...*/
669 DEBUG(0,("some_string: %s\n",data->some_string));
671 return SMB_VFS_NEXT_CLOSE(handle, fsp, fd);
674 second way of doing it:
676 static void free_example_privates(void **datap)
678 struct example_privates *data = (struct example_privates *)*datap;
680 SAFE_FREE(data->some_string);
688 static int example_connect(vfs_handle_struct *handle,
689 connection_struct *conn, const char *service,
692 struct example_privates *data = NULL;
694 /* alloc our private data */
695 data = (struct example_privates *)malloc(sizeof(struct example_privates));
697 DEBUG(0,("malloc() failed\n"));
701 /* init out private data */
702 data->some_string = strdup("test");
703 if (!data->some_string) {
704 DEBUG(0,("strdup() failed\n"));
708 data->db_connection = open_db_conn();
710 /* and now store the private data pointer in handle->data
711 * we need to specify a free_function because we used malloc() and strdup().
712 * (return -1 if something failed.)
714 SMB_VFS_HANDLE_SET_DATA(handle, data, free_example_privates, struct example_privates, return -1);
716 return SMB_VFS_NEXT_CONNECT(handle,conn,service,user);
719 static int example_close(vfs_handle_struct *handle, files_struct *fsp, int fd)
721 struct example_privates *data = NULL;
723 /* get the pointer to our private data
724 * return -1 if something failed
726 SMB_VFS_HANDLE_GET_DATA(handle, data, struct example_privates, return -1);
728 /* do something here...*/
729 DEBUG(0,("some_string: %s\n",data->some_string));
731 return SMB_VFS_NEXT_CLOSE(handle, fsp, fd);
737 To make it easy to build 3rd party modules it would be usefull to provide
738 configure.in, (configure), install.sh and Makefile.in with the module.
739 (Take a look at the example in <filename>examples/VFS</filename>.)
743 The configure script accepts <option>--with-samba-source</option> to specify
744 the path to the samba source tree.
745 It also accept <option>--enable-developer</option> which lets the compiler
746 give you more warnings.
750 The idea is that you can extend this
751 <filename>configure.in</filename> and <filename>Makefile.in</filename> scripts
756 Compiling & Testing...
758 <member><userinput>./configure <option>--enable-developer</option></userinput> ...</member>
759 <member><userinput>make</userinput></member>
760 <member>Try to fix all compiler warnings</member>
761 <member><userinput>make</userinput></member>
762 <member>Testing, Testing, Testing ...</member>
771 <title>Some Notes</title>
774 <title>Implement TRANSPARENT functions</title>
777 Avoid writing functions like this:
780 static int example_close(vfs_handle_struct *handle, files_struct *fsp, int fd)
782 return SMB_VFS_NEXT_CLOSE(handle, fsp, fd);
786 Overload only the functions you really need to!
792 <title>Implement OPAQUE functions</title>
795 If you want to just implement a better version of a
796 default samba opaque function
797 (e.g. like a disk_free() function for a special filesystem)
798 it's ok to just overload that specific function.
802 If you want to implement a database filesystem or
803 something different from a posix filesystem.
804 Make sure that you overload every vfs operation!!!
807 Functions your FS does not support should be overloaded by something like this:
808 e.g. for a readonly filesystem.
812 static int example_rename(vfs_handle_struct *handle, connection_struct *conn,
813 char *oldname, char *newname)
815 DEBUG(10,("function rename() not allowed on vfs 'example'\n"));