3 ###################################################
4 # package to parse IDL files and generate code for
5 # rpc functions in Samba
6 # Copyright tridge@samba.org 2000-2003
7 # Copyright jelmer@samba.org 2005-2007
8 # released under the GNU GPL
14 pidl - An IDL compiler written in Perl
20 pidl [--outputdir[=OUTNAME]] [--includedir DIR...] [--parse-idl-tree] [--dump-idl-tree] [--dump-ndr-tree] [--header[=OUTPUT]] [--python[=OUTPUT]] [--ndr-parser[=OUTPUT]] [--client] [--server] [--warn-compat] [--quiet] [--verbose] [--template] [--ws-parser[=OUTPUT]] [--diff] [--dump-idl] [--tdr-parser[=OUTPUT]] [--samba3-ndr-client[=OUTPUT]] [--samba3-ndr-server[=OUTPUT]] [--typelib=[OUTPUT]] [<idlfile>.idl]...
24 pidl is an IDL compiler written in Perl that aims to be somewhat
25 compatible with the midl compiler. IDL is short for
26 "Interface Definition Language".
28 pidl can generate stubs for DCE/RPC server code, DCE/RPC
29 client code and Wireshark dissectors for DCE/RPC traffic.
31 IDL compilers like pidl take a description
32 of an interface as their input and use it to generate C
33 (though support for other languages may be added later) code that
34 can use these interfaces, pretty print data sent
35 using these interfaces, or even generate Wireshark
36 dissectors that can parse data sent over the
37 wire by these interfaces.
39 pidl takes IDL files in the same format as is used by midl,
40 converts it to a .pidl file (which contains pidl's internal representation of the interface) and can then generate whatever output you need.
41 .pidl files should be used for debugging purposes only. Write your
42 interface definitions in .idl format.
44 The goal of pidl is to implement a IDL compiler that can be used
45 while developing the RPC subsystem in Samba (for
46 both marshalling/unmarshalling and debugging purposes).
54 Show list of available options.
60 =item I<--outputdir OUTNAME>
62 Write output files to the specified directory. Defaults to the current
65 =item I<--includedir DIR>
67 Add DIR to the search path used by the preprocessor. This option can be
68 specified multiple times.
70 =item I<--parse-idl-tree>
72 Read internal tree structure from input files rather
73 than assuming they contain IDL.
77 Generate a new IDL file. File will be named OUTNAME.idl.
81 Generate a C header file for the specified interface. Filename defaults to OUTNAME.h.
85 Generate a C file and C header containing NDR parsers. The filename for
86 the parser defaults to ndr_OUTNAME.c. The header filename will be the
87 parser filename with the extension changed from .c to .h.
91 Generate a C file and C header containing TDR parsers. The filename for
92 the parser defaults to tdr_OUTNAME.c. The header filename will be the
93 parser filename with the extension changed from .c to .h.
97 Write type information to the specified file.
101 Generate boilerplate for the RPC server that implements
102 the interface. Filename defaults to ndr_OUTNAME_s.c.
106 Generate stubs for a RPC server that implements the interface. Output will
107 be written to stdout.
111 Generate an Wireshark dissector (in C) and header file. The dissector filename
112 defaults to packet-dcerpc-OUTNAME.c while the header filename defaults to
113 packet-dcerpc-OUTNAME.h.
115 Pidl will read additional data from an Wireshark conformance file if present.
116 Such a file should have the same location as the IDL file but with the
117 extension I<cnf> rather than I<idl>. See L<Parse::Pidl::Wireshark::Conformance>
118 for details on the format of this file.
122 Parse an IDL file, generate a new IDL file based on the internal data
123 structures and see if there are any differences with the original IDL file.
124 Useful for debugging pidl.
126 =item I<--dump-idl-tree>
128 Tell pidl to dump the internal tree representation of an IDL
129 file the to disk. Useful for debugging pidl.
131 =item I<--dump-ndr-tree>
133 Tell pidl to dump the internal NDR information tree it generated
134 from the IDL file to disk. Useful for debugging pidl.
136 =item I<--samba3-ndr-client>
138 Generate client calls for Samba3, to be placed in rpc_client/. Instead of
139 calling out to the code in Samba3's rpc_parse/, this will call out to
140 Samba4's NDR code instead.
142 =item I<--samba3-ndr-server>
144 Generate server calls for Samba3, to be placed in rpc_server/. Instead of
145 calling out to the code in Samba3's rpc_parse/, this will call out to
146 Samba4's NDR code instead.
152 IDL files are always preprocessed using the C preprocessor.
154 Pretty much everything in an interface (the interface itself, functions,
155 parameters) can have attributes (or properties whatever name you give them).
156 Attributes always prepend the element they apply to and are surrounded
157 by square brackets ([]). Multiple attributes are separated by comma's;
158 arguments to attributes are specified between parentheses.
160 See the section COMPATIBILITY for the list of attributes that
163 C-style comments can be used.
165 =head2 CONFORMANT ARRAYS
167 A conformant array is one with that ends in [*] or []. The strange
168 things about conformant arrays are that they can only appear as the last
169 element of a structure (unless there is a pointer to the conformant array,
170 of course) and the array size appears before the structure itself on the wire.
178 [size_is(count)] long s[*];
181 it appears like this:
183 [size_is] [abc] [count] [foo] [s...]
185 the first [size_is] field is the allocation size of the array, and
186 occurs before the array elements and even before the structure
189 Note that size_is() can refer to a constant, but that doesn't change
190 the wire representation. It does not make the array a fixed array.
192 midl.exe would write the above array as the following C header:
201 pidl takes a different approach, and writes it like this:
210 =head2 VARYING ARRAYS
212 A varying array looks like this:
218 [size_is(count)] long *s;
221 This will look like this on the wire:
223 [abc] [count] [foo] [PTR_s] [count] [s...]
227 A fixed array looks like this:
233 The NDR representation looks just like 10 separate long
234 declarations. The array size is not encoded on the wire.
236 pidl also supports "inline" arrays, which are not part of the IDL/NDR
237 standard. These are declared like this:
246 This appears like this:
248 [foo] [count] [bar] [s...]
250 Fixed arrays are an extension added to support some of the strange
251 embedded structures in security descriptors and spoolss.
253 This section is by no means complete. See the OpenGroup and MSDN
254 documentation for additional information.
256 =head1 COMPATIBILITY WITH MIDL
258 =head2 Missing features in pidl
260 The following MIDL features are not (yet) implemented in pidl
261 or are implemented with an incompatible interface:
267 Asynchronous communication
271 Typelibs (.tlb files)
275 Datagram support (ncadg_*)
279 =head2 Supported attributes and statements
281 in, out, ref, length_is, switch_is, size_is, uuid, case, default, string,
282 unique, ptr, pointer_default, v1_enum, object, helpstring, range, local,
283 call_as, endpoint, switch_type, progid, coclass, iid_is, represent_as,
284 transmit_as, import, include, cpp_quote.
286 =head2 PIDL Specific properties
292 The [public] property on a structure or union is a pidl extension that
293 forces the generated pull/push functions to be non-static. This allows
294 you to declare types that can be used between modules. If you don't
295 specify [public] then pull/push functions for other than top-level
296 functions are declared static.
300 The [noprint] property is a pidl extension that allows you to specify
301 that pidl should not generate a ndr_print_*() function for that
302 structure or union. This is used when you wish to define your own
303 print function that prints a structure in a nicer manner. A good
304 example is the use of [noprint] on dom_sid, which allows the
305 pretty-printing of SIDs.
309 The [value(expression)] property is a pidl extension that allows you
310 to specify the value of a field when it is put on the wire. This
311 allows fields that always have a well-known value to be automatically
312 filled in, thus making the API more programmer friendly. The
313 expression can be any C expression.
317 The [relative] property can be supplied on a pointer. When it is used
318 it declares the pointer as a spoolss style "relative" pointer, which
319 means it appears on the wire as an offset within the current
320 encapsulating structure. This is not part of normal IDL/NDR, but it is
321 a very useful extension as it avoids the manual encoding of many
324 =item subcontext(length)
326 Specifies that a size of I<length>
327 bytes should be read, followed by a blob of that size,
328 which will be parsed as NDR.
330 subcontext() is deprecated now, and should not be used in new code.
331 Instead, use represent_as() or transmit_as().
335 Specify boolean options, mostly used for
336 low-level NDR options. Several options
337 can be specified using the | character.
338 Note that flags are inherited by substructures!
342 The [nodiscriminant] property on a union means that the usual uint16
343 discriminent field at the start of the union on the wire is
344 omitted. This is not normally allowed in IDL/NDR, but is used for some
349 Specify that the array or string uses the specified
350 charset. If this attribute is specified, pidl will
351 take care of converting the character data from this format
352 to the host format. Commonly used values are UCS2, DOS and UTF8.
356 =head2 Unsupported MIDL properties or statements
358 aggregatable, appobject, async_uuid, bindable, control,
359 defaultbind, defaultcollelem, defaultvalue, defaultvtable, dispinterface,
360 displaybind, dual, entry, first_is, helpcontext, helpfile, helpstringcontext,
361 helpstringdll, hidden, idl_module, idl_quote, id, immediatebind, importlib,
362 includelib, last_is, lcid, licensed, max_is, module,
363 ms_union, no_injected_text, nonbrowsable, noncreatable, nonextensible, odl,
364 oleautomation, optional, pragma, propget, propputref, propput, readonly,
365 requestedit, restricted, retval, source, uidefault,
366 usesgetlasterror, vararg, vi_progid, wire_marshal.
370 # Generating an Wireshark parser
371 $ ./pidl --ws-parser -- atsvc.idl
373 # Generating a TDR parser and header
374 $ ./pidl --tdr-parser --header -- regf.idl
376 # Generating a Samba3 client and server
377 $ ./pidl --samba3-ndr-client --samba3-ndr-server -- dfs.idl
379 # Generating a Samba4 NDR parser, client and server
380 $ ./pidl --ndr-parser --ndr-client --ndr-server -- samr.idl
384 L<https://msdn.microsoft.com/en-us/library/windows/desktop/aa373864%28v=vs.85%29.aspx>
385 L<https://wiki.wireshark.org/DCE/RPC>,
386 L<https://www.samba.org/>,
391 pidl is licensed under the GNU General Public License L<https://www.gnu.org/licenses/gpl.html>.
395 pidl was written by Andrew Tridgell, Stefan Metzmacher, Tim Potter and Jelmer
396 Vernooij. The current maintainer is Jelmer Vernooij.
398 This manpage was written by Jelmer Vernooij, partially based on the original
399 pidl README by Andrew Tridgell.
405 use FindBin qw($RealBin $Script);
406 use lib "$RealBin/lib";
409 use Parse::Pidl qw ( $VERSION );
410 use Parse::Pidl::Util;
411 use Parse::Pidl::ODL;
413 #####################################################################
414 # save a data structure into a file
415 sub SaveStructure($$)
417 my($filename,$v) = @_;
418 FileSave($filename, Parse::Pidl::Util::MyDumper($v));
421 #####################################################################
422 # load a data structure from a file (as saved with SaveStructure)
426 my $contents = FileLoad($f);
427 defined $contents || return undef;
428 return eval "$contents";
431 #####################################################################
432 # read a file into a string
435 my($filename) = shift;
437 open(INPUTFILE, $filename) || return undef;
438 my($saved_delim) = $/;
440 my($data) = <INPUTFILE>;
446 #####################################################################
447 # write a string into a file
450 my($filename) = shift;
453 open(FILE, ">$filename") || die "can't open $filename";
458 my(@opt_incdirs) = ();
460 my($opt_version) = 0;
461 my($opt_parse_idl_tree) = 0;
462 my($opt_dump_idl_tree);
463 my($opt_dump_ndr_tree);
464 my($opt_dump_idl) = 0;
467 my($opt_samba3_header);
468 my($opt_samba3_parser);
469 my($opt_samba3_server);
470 my($opt_samba3_ndr_client);
471 my($opt_samba3_ndr_server);
472 my($opt_samba3_compat_server) = 0;
473 my($opt_samba3_template) = 0;
474 my($opt_template) = 0;
483 my($opt_outputdir) = '.';
484 my($opt_verbose) = 0;
485 my($opt_warn_compat) = 0;
489 #########################################
493 print "perl IDL parser and code generator\n";
496 Copyright (C) Andrew Tridgell <tridge\@samba.org>
497 Copyright (C) Jelmer Vernooij <jelmer\@samba.org>
499 Usage: $Script [options] [--] <idlfile> [<idlfile>...]
502 --help this help page
503 --version show pidl version
504 --outputdir=OUTDIR put output in OUTDIR/ [.]
505 --warn-compat warn about incompatibility with other compilers
508 --includedir DIR search DIR for included files
511 --dump-idl-tree[=FILE] dump internal representation to file [BASENAME.pidl]
512 --parse-idl-tree read internal representation instead of IDL
513 --dump-ndr-tree[=FILE] dump internal NDR data tree to file [BASENAME.ndr]
514 --dump-idl regenerate IDL file
515 --diff run diff on original IDL and dumped output
516 --typelib print type information
519 --header[=OUTFILE] create generic header file [BASENAME.h]
520 --ndr-parser[=OUTFILE] create a C NDR parser [ndr_BASENAME.c]
521 --client[=OUTFILE] create a C NDR client [ndr_BASENAME_c.c]
522 --tdr-parser[=OUTFILE] create a C TDR parser [tdr_BASENAME.c]
523 --python[=OUTFILE] create python wrapper file [py_BASENAME.c]
524 --server[=OUTFILE] create server boilerplate [ndr_BASENAME_s.c]
525 --template print a template for a pipe
526 --dcom-proxy[=OUTFILE] create DCOM proxy [ndr_BASENAME_p.c]
527 --com-header[=OUTFILE] create header for COM [com_BASENAME.h]
530 --samba3-ndr-client[=OUTF] create client calls for Samba3
531 using Samba4's NDR code [cli_BASENAME.c]
532 --samba3-ndr-server[=OUTF] create server call wrapper for Samba3
533 using Samba4's NDR code [srv_BASENAME.c]
534 --samba3-template print a template for a pipe
537 --ws-parser[=OUTFILE] create Wireshark parser and header
542 #########################################
546 print "perl IDL version $VERSION\n";
550 my $result = GetOptions (
551 'help|h|?' => \$opt_help,
552 'version' => \$opt_version,
553 'outputdir=s' => \$opt_outputdir,
554 'dump-idl' => \$opt_dump_idl,
555 'dump-idl-tree:s' => \$opt_dump_idl_tree,
556 'parse-idl-tree' => \$opt_parse_idl_tree,
557 'dump-ndr-tree:s' => \$opt_dump_ndr_tree,
558 'samba3-ndr-client:s' => \$opt_samba3_ndr_client,
559 'samba3-ndr-server:s' => \$opt_samba3_ndr_server,
560 'samba3-compat-server:s' => \$opt_samba3_compat_server,
561 'samba3-template' => \$opt_samba3_template,
562 'header:s' => \$opt_header,
563 'server:s' => \$opt_server,
564 'typelib:s' => \$opt_typelib,
565 'tdr-parser:s' => \$opt_tdr_parser,
566 'template' => \$opt_template,
567 'ndr-parser:s' => \$opt_ndr_parser,
568 'client:s' => \$opt_client,
569 'ws-parser:s' => \$opt_ws_parser,
570 'python' => \$opt_python,
571 'diff' => \$opt_diff,
572 'dcom-proxy:s' => \$opt_dcom_proxy,
573 'com-header:s' => \$opt_com_header,
574 'quiet' => \$opt_quiet,
575 'verbose' => \$opt_verbose,
576 'warn-compat' => \$opt_warn_compat,
577 'includedir=s@' => \@opt_incdirs
594 $opt_dump_idl_tree = "";
595 $opt_dump_ndr_tree = "";
599 my $idl_file = shift;
600 my $outputdir = $opt_outputdir;
604 my $basename = basename($idl_file, ".idl");
606 unless ($opt_quiet) { print "Compiling $idl_file\n"; }
608 if ($opt_parse_idl_tree) {
609 $pidl = LoadStructure($idl_file);
610 defined $pidl || die "Failed to load $idl_file";
612 require Parse::Pidl::IDL;
614 $pidl = Parse::Pidl::IDL::parse_file($idl_file, \@opt_incdirs);
615 defined $pidl || die "Failed to parse $idl_file";
618 require Parse::Pidl::Typelist;
619 Parse::Pidl::Typelist::LoadIdl($pidl, $basename);
621 if (defined($opt_dump_idl_tree)) {
622 my($pidl_file) = ($opt_dump_idl_tree or "$outputdir/$basename.pidl");
623 SaveStructure($pidl_file, $pidl) or die "Failed to save $pidl_file\n";
627 require Parse::Pidl::Dump;
628 print Parse::Pidl::Dump($pidl);
632 my($tempfile) = "$outputdir/$basename.tmp";
633 FileSave($tempfile, IdlDump::Dump($pidl));
634 system("diff -wu $idl_file $tempfile");
638 my $comh_filename = ($opt_com_header or "$outputdir/com_$basename.h");
639 if (defined($opt_com_header)) {
640 require Parse::Pidl::Samba4::COM::Header;
641 my $res = Parse::Pidl::Samba4::COM::Header::Parse($pidl,"$outputdir/ndr_$basename.h");
643 FileSave($comh_filename, $res);
647 if (defined($opt_dcom_proxy)) {
648 require Parse::Pidl::Samba4::COM::Proxy;
649 my $res = Parse::Pidl::Samba4::COM::Proxy::Parse($pidl,$comh_filename);
651 my ($client) = ($opt_dcom_proxy or "$outputdir/$basename\_p.c");
652 FileSave($client, $res);
656 if ($opt_warn_compat) {
657 require Parse::Pidl::Compat;
658 Parse::Pidl::Compat::Check($pidl);
661 $pidl = Parse::Pidl::ODL::ODL2IDL($pidl, dirname($idl_file), \@opt_incdirs);
663 if (defined($opt_ws_parser)) {
664 require Parse::Pidl::Wireshark::NDR;
666 my $cnffile = $idl_file;
667 $cnffile =~ s/\.idl$/\.cnf/;
669 my $generator = new Parse::Pidl::Wireshark::NDR();
670 $generator->Initialize($cnffile);
674 if (defined($opt_ws_parser) or
675 defined($opt_client) or
676 defined($opt_server) or
677 defined($opt_header) or
678 defined($opt_ndr_parser) or
679 defined($opt_python) or
680 defined($opt_dump_ndr_tree) or
681 defined($opt_samba3_header) or
682 defined($opt_samba3_parser) or
683 defined($opt_samba3_server) or
684 defined($opt_samba3_ndr_client) or
685 defined($opt_samba3_ndr_server) or
686 defined($opt_samba3_compat_server)) {
687 require Parse::Pidl::NDR;
688 $ndr = Parse::Pidl::NDR::Parse($pidl);
691 if (defined($opt_dump_ndr_tree)) {
692 my($ndr_file) = ($opt_dump_ndr_tree or "$outputdir/$basename.ndr");
693 SaveStructure($ndr_file, $ndr) or die "Failed to save $ndr_file\n";
696 my $gen_header = ($opt_header or "$outputdir/$basename.h");
697 if (defined($opt_header)) {
698 require Parse::Pidl::Samba4::Header;
699 FileSave($gen_header, Parse::Pidl::Samba4::Header::Parse($ndr));
702 my $h_filename = "$outputdir/ndr_$basename.h";
703 my $c_header = "$outputdir/ndr_$basename\_c.h";
704 if (defined($opt_client) or defined($opt_samba3_ndr_client)) {
705 require Parse::Pidl::Samba4::NDR::Client;
706 my ($c_client) = ($opt_client or "$outputdir/ndr_$basename\_c.c");
707 $c_header = $c_client;
708 $c_header =~ s/\.c$/.h/;
710 my $generator = new Parse::Pidl::Samba4::NDR::Client();
711 my ($srcd,$hdrd) = $generator->Parse(
712 $ndr,$gen_header,$h_filename,$c_header);
714 FileSave($c_client, $srcd);
715 FileSave($c_header, $hdrd);
718 if (defined($opt_python)) {
719 require Parse::Pidl::Samba4::Python;
720 my $generator = new Parse::Pidl::Samba4::Python();
721 my ($prsr) = $generator->Parse($basename, $ndr,
722 "$outputdir/ndr_$basename\_c.h", $h_filename);
723 FileSave("$outputdir/py_$basename.c", $prsr);
726 if (defined($opt_server)) {
727 require Parse::Pidl::Samba4::NDR::Server;
729 FileSave(($opt_server or "$outputdir/ndr_$basename\_s.c"), Parse::Pidl::Samba4::NDR::Server::Parse($ndr,$h_filename));
732 if (defined($opt_ndr_parser)) {
733 my $parser_fname = ($opt_ndr_parser or "$outputdir/ndr_$basename.c");
734 require Parse::Pidl::Samba4::NDR::Parser;
735 my $generator = new Parse::Pidl::Samba4::NDR::Parser();
736 my ($header,$parser) = $generator->Parse($ndr, $gen_header, $h_filename);
738 FileSave($parser_fname, $parser);
739 FileSave($h_filename, $header);
743 if (defined($opt_ws_parser)) {
744 require Parse::Pidl::Wireshark::NDR;
745 my($eparser) = ($opt_ws_parser or "$outputdir/packet-dcerpc-$basename.c");
746 my $eheader = $eparser;
747 $eheader =~ s/\.c$/\.h/;
748 my $cnffile = $idl_file;
749 $cnffile =~ s/\.idl$/\.cnf/;
751 my $generator = new Parse::Pidl::Wireshark::NDR();
752 my ($dp, $dh) = $generator->Parse($ndr, $idl_file, $eheader, $cnffile);
753 FileSave($eparser, $dp) if defined($dp);
754 FileSave($eheader, $dh) if defined($dh);
757 if (defined($opt_tdr_parser)) {
758 my $tdr_parser = ($opt_tdr_parser or "$outputdir/tdr_$basename.c");
759 my $tdr_header = $tdr_parser;
760 $tdr_header =~ s/\.c$/\.h/;
761 require Parse::Pidl::Samba4::TDR;
762 my $generator = new Parse::Pidl::Samba4::TDR();
763 my ($hdr,$prsr) = $generator->Parser($pidl, $tdr_header, $gen_header);
764 FileSave($tdr_parser, $prsr);
765 FileSave($tdr_header, $hdr);
768 if (defined($opt_typelib)) {
769 my $typelib = ($opt_typelib or "$outputdir/$basename.tlb");
770 require Parse::Pidl::Typelist;
771 FileSave($typelib, Parse::Pidl::Typelist::GenerateTypeLib());
775 require Parse::Pidl::Samba4::Template;
776 print Parse::Pidl::Samba4::Template::Parse($pidl);
779 if ($opt_samba3_template) {
780 require Parse::Pidl::Samba3::Template;
781 print Parse::Pidl::Samba3::Template::Parse($pidl);
784 if (defined($opt_samba3_ndr_client)) {
785 my $client = ($opt_samba3_ndr_client or "$outputdir/cli_$basename.c");
786 my $header = $client; $header =~ s/\.c$/\.h/;
787 require Parse::Pidl::Samba3::ClientNDR;
788 my $generator = new Parse::Pidl::Samba3::ClientNDR();
789 my ($c_code,$h_code) = $generator->Parse($ndr, $header, $c_header);
790 FileSave($client, $c_code);
791 FileSave($header, $h_code);
794 if (defined($opt_samba3_ndr_server)) {
795 my $server = ($opt_samba3_ndr_server or "$outputdir/srv_$basename.c");
796 my $header = $server; $header =~ s/\.c$/\.h/;
797 require Parse::Pidl::Samba3::ServerNDR;
798 my ($c_code,$h_code) = Parse::Pidl::Samba3::ServerNDR::Parse($ndr, $header, $h_filename);
799 FileSave($server, $c_code);
800 FileSave($header, $h_code);
803 if (defined($opt_samba3_compat_server)) {
804 my $server = ($opt_samba3_compat_server or "$outputdir/s3compat_$basename.c");
805 my $header = $server; $header =~ s/\.c$/\.h/;
806 require Parse::Pidl::Samba3::CompatServer;
807 my ($c_code,$h_code) = Parse::Pidl::Samba3::CompatServer::Parse($ndr, $header, $h_filename);
808 FileSave($server, $c_code);
809 FileSave($header, $h_code);
813 if (scalar(@ARGV) == 0) {
814 print "$Script: no input files\n";
818 process_file($_) foreach (@ARGV);