diff options
Diffstat (limited to 'doc/src/FAQ/FAQ_DEV.html')
| -rw-r--r-- | doc/src/FAQ/FAQ_DEV.html | 486 |
1 files changed, 486 insertions, 0 deletions
diff --git a/doc/src/FAQ/FAQ_DEV.html b/doc/src/FAQ/FAQ_DEV.html new file mode 100644 index 00000000000..ba60c157c21 --- /dev/null +++ b/doc/src/FAQ/FAQ_DEV.html @@ -0,0 +1,486 @@ +<HTML> +<HEAD> +<TITLE>PostgreSQL Developers FAQ</title> +</HEAD> +<BODY BGCOLOR="#FFFFFF" TEXT="#000000" LINK="#FF0000" VLINK="#A00000" ALINK="#0000FF"> +<H1> +Developer's Frequently Asked Questions (FAQ) for PostgreSQL +</H1> +<P> +Last updated: Fri Jun 9 21:54:54 EDT 2000 +<P> +Current maintainer: Bruce Momjian (<a +href="mailto:pgman@candle.pha.pa.us">pgman@candle.pha.pa.us</a>)<BR> +<P> +The most recent version of this document can be viewed at +the postgreSQL Web site, <a +href="http://PostgreSQL.org">http://PostgreSQL.org</a>. +<P> +<HR> +<P> + +<CENTER><H2>Questions</H2></CENTER> +<a href="#1">1</a>) What tools are available for developers?<BR> +<a href="#2">2</a>) What books are good for developers?<BR> +<a href="#3">3</a>) Why do we use <I>palloc</I>() and <I>pfree</I>() to allocate memory?<BR> +<a href="#4">4</a>) Why do we use <I>Node</I> and <I>List</I> to +make data structures?<BR> +<a href="#5">5</a>) How do I add a feature or fix a bug?<BR> +<a href="#6">6</a>) How do I download/update the current source tree?<BR> +<a href="#7">7</a>) How do I test my changes?<BR> +<a href="#7">7</a>) I just added a field to a structure. What else +should I do?<BR> +<a href="#8">8</a>) Why are table, column, type, function, view +names sometimes referenced as <I>Name</I> or <I>NameData,</I> and +sometimes as <I>char *?</I><BR> +<a href="#9">9</a>) How do I efficiently access information in +tables from the backend code?<BR> +<a href="#10">10</a>) What is elog()?<BR> +<a href="#11">11</a>) What is configure all about?<BR> +<a href="#12">12</a>) How do I add a new port?<BR> +<BR> +<HR> + +<H3><a +name="1">1</a>) What tools are available for developers?</H3><P> + +Aside from the User documentation mentioned in the regular FAQ, there +are several development tools available. First, all the files in the +<I>/tools</I> directory are designed for developers. + +<PRE> + RELEASE_CHANGES changes we have to make for each release + SQL_keywords standard SQL'92 keywords + backend description/flowchart of the backend directories + ccsym find standard defines made by your compiler + entab converts tabs to spaces, used by pgindent + find_static finds functions that could be made static + find_typedef get a list of typedefs in the source code + make_ctags make vi 'tags' file in each directory + make_diff make *.orig and diffs of source + make_etags make emacs 'etags' files + make_keywords.README make comparison of our keywords and SQL'92 + make_mkid make mkid ID files + mkldexport create AIX exports file + pgindent indents C source files + pginclude scripts for adding/removing include files + unused_oids in pgsql/src/include/catalog +</PRE> + +Let me note some of these. If you point your browser at the +<I>file:/usr/local/src/pgsql/src/tools/backend/index.html</I> directory, +you will see few paragraphs describing the data flow, the backend +components in a flow chart, and a description of the shared memory area. +You can click on any flowchart box to see a description. If you then +click on the directory name, you will be taken to the source directory, +to browse the actual source code behind it. We also have several README +files in some source directories to describe the function of the module. + The browser will display these when you enter the directory also. The +<I>tools/backend</I> directory is also contained on our web page under +the title <I>How PostgreSQL Processes a Query.</I><P> + + +Second, you really should have an editor that can handle tags, so you +can tag a function call to see the function definition, and then tag +inside that function to see an even lower-level function, and then back +out twice to return to the original function. Most editors support this +via <I>tags</I> or <I>etags</I> files.<P> + + +Third, you need to get <I>id-utils</I> from: +<pre> + <a href="ftp://alpha.gnu.org/gnu/id-utils-3.2d.tar.gz">ftp://alpha.gnu.org/gnu/id-utils-3.2d.tar.gz</a> + <a href="ftp://tug.org/gnu/id-utils-3.2d.tar.gz">ftp://tug.org/gnu/id-utils-3.2d.tar.gz</a> + <a href="ftp://ftp.enst.fr/pub/gnu/gnits/id-utils-3.2d.tar.gz">ftp://ftp.enst.fr/pub/gnu/gnits/id-utils-3.2d.tar.gz</a> +</pre> + +By running <I>tools/make_mkid</I>, an archive of source symbols can be +created that can be rapidly queried like <I>grep</I> or edited. Others +prefer <I>glimpse.</I><P> + + +<I>make_diff</I> has tools to create patch diff files that can be +applied to the distribution.<P> + + +Our standard format is to indent each code level with one tab, where +each tab is four spaces. You will need to set your editor to display +tabs as four spaces: +<BR> +<PRE> + vi in ~/.exrc: + set tabstop=4 + set sw=4 + more: + more -x4 + less: + less -x4 + emacs: + M-x set-variable tab-width + or + ; Cmd to set tab stops &etc for working with PostgreSQL code + (c-add-style "pgsql" + '("bsd" + (indent-tabs-mode . t) + (c-basic-offset . 4) + (tab-width . 4) + (c-offsets-alist . + ((case-label . +)))) + t) ; t = set this mode on + + and add this to your autoload list (modify file path in macro): + + (setq auto-mode-alist + (cons '("\\`/usr/local/src/pgsql/.*\\.[chyl]\\'" . pgsql-c-mode) + auto-mode-alist)) + or + /* + * Local variables: + * tab-width: 4 + * c-indent-level: 4 + * c-basic-offset: 4 + * End: + */ +</PRE> +<BR> +<I>pgindent</I> will the format code by specifying +flags to your operating system's utility <I>indent.</I><P> +<I>pgindent</I> is run on all source files just before each beta test +period. It auto-formats all source files to make them consistent. +Comment blocks that need specific line breaks should be formatted as +<I>block comments,</I> where the comment starts as +<CODE>/*------</CODE>. These comments will not be reformatted in any +way. + +<I>pginclude</I> contains scripts used to add needed #include's to +include files, and removed unneeded #include's. + +When adding system types, you will need to assign oids to them. +There is also a script called <I>unused_oids</I> in +<I>pgsql/src/include/catalog</I> that shows the unused oids. + +<H3><a name="2">2</a>) What books are good for developers?</H3><P> + +I have four good books, <I>An Introduction to Database Systems,</I> by +C.J. Date, Addison, Wesley, <I>A Guide to the SQL Standard,</I> by C.J. +Date, et. al, Addison, Wesley, <I>Fundamentals of Database Systems,</I> +by Elmasri and Navathe, and <I>Transaction Processing,</I> by Jim Gray, +Morgan, Kaufmann<P> + +There is also a database performance site, with a handbook on-line +written by Jim Gray at <A +HREF="http://www.benchmarkresources.com">http://www.benchmarkresources.com.</A> + + + +<H3><a name="3">3</a>) Why do we use <I>palloc</I>() and <I>pfree</I>() +to allocate memory?</H3><P> + +<I>palloc()</I> and <I>pfree()</I> are used in place of malloc() and +free() because we automatically free all memory allocated when a +transaction completes. This makes it easier to make sure we free memory +that gets allocated in one place, but only freed much later. There are +several contexts that memory can be allocated in, and this controls when +the allocated memory is automatically freed by the backend.<P> + + +<H3><a name="4">4</a>) Why do we use <I>Node</I> and <I>List</I> to +make data structures?</H3><P> + +We do this because this allows a consistent way to pass data inside the +backend in a flexible way. Every node has a <I>NodeTag</I> which +specifies what type of data is inside the Node. <I>Lists</I> are groups +of <I>Nodes chained together as a forward-linked list.</I><P> +Here are some of the <I>List</I> manipulation commands: +<BLOCKQUOTE> +<DL> +<DT>lfirst(i) +<DD>return the data at list element <I>i.</I> +<DT>lnext(i) +<DD>return the next list element after <I>i.</I> +<DT>foreach(i, list) +<DD>loop through <I>list,</I> assigning each list element to <I>i.</I> +It is important to note that <I>i</I> is a List *, not the data in the +<I>List</I> element. You need to use <I>lfirst(i)</I> to get at the data. +Here is a typical code snipped that loops through a List containing +<I>Var *'s</I> and processes each one: +<PRE> +<CODE> + List *i, *list; + + foreach(i, list) + { + Var *var = lfirst(i); + + /* process var here */ + } +</CODE> +</PRE> +<DT>lcons(node, list) +<DD>add <I>node</I> to the front of <I>list,</I> or create a new list with +<I>node</I> if <I>list</I> is <I>NIL.</I> +<DT>lappend(list, node) +<DD>add <I>node</I> to the end of <I>list.</I> This is more expensive +that lcons. +<DT>nconc(list1, list2) +<DD>Concat <I>list2</I> on to the end of <I>list1.</I> +<DT>length(list) +<DD>return the length of the <I>list.</I> +<DT>nth(i, list) +<DD>return the <I>i</I>'th element in <I>list.</I> +<DT>lconsi, ... +<DD>There are integer versions of these: <I>lconsi, lappendi, nthi.</I> +<I>List's</I> containing integers instead of Node pointers are used to +hold list of relation object id's and other integer quantities. +</DL> +</BLOCKQUOTE> +You can print nodes easily inside <I>gdb.</I> First, to disable +output truncation when you use the gdb <I>print</I> command: +<PRE> +<CODE> + (gdb) set print elements 0 +</CODE> +</PRE> +Instead of printing values in gdb format, you can use the next two +commands to print out List, Node, and structure contents in a verbose +format that is easier to understand. List's are unrolled into nodes, +and nodes are printed in detail. The first prints in a short format, +and the second in a long format: +<PRE> +<CODE> + (gdb) call print(any_pointer) + (gdb) call pprint(any_pointer) +</CODE> +</PRE> +The output appears in the postmaster log file, or on your screen if you +are running a backend directly without a postmaster. +<P> + +<H3><a name="5">5</a>) How do I add a feature or fix a bug?</H3><P> + +The source code is over 250,000 lines. Many problems/features are +isolated to one specific area of the code. Others require knowledge of +much of the source. If you are confused about where to start, ask the +hackers list, and they will be glad to assess the complexity and give +pointers on where to start.<P> + +Another thing to keep in mind is that many fixes and features can be +added with surprisingly little code. I often start by adding code, then +looking at other areas in the code where similar things are done, and by +the time I am finished, the patch is quite small and compact.<P> + +When adding code, keep in mind that it should use the existing +facilities in the source, for performance reasons and for simplicity. +Often a review of existing code doing similar things is helpful.<P> + + +<H3><a name="6">6</a>) How do I download/update the current source +tree?</H3><P> + + +There are several ways to obtain the source tree. Occasional developers +can just get the most recent source tree snapshot from +ftp.postgresql.org. For regular developers, you can use CVS. CVS +allows you to download the source tree, then occasionally update your +copy of the source tree with any new changes. Using CVS, you don't have +to download the entire source each time, only the changed files. +Anonymous CVS does not allows developers to update the remote source +tree, though privileged developers can do this. There is a CVS FAQ on +our web site that describes how to use remote CVS. You can also use +CVSup, which has similarly functionality, and is available from +ftp.postgresql.org.<P> + +To update the source tree, there are two ways. You can generate a patch +against your current source tree, perhaps using the make_diff tools +mentioned above, and send them to the patches list. They will be +reviewed, and applied in a timely manner. If the patch is major, and we +are in beta testing, the developers may wait for the final release +before applying your patches.<P> + +For hard-core developers, Marc(scrappy@postgresql.org) will give you a +Unix shell account on postgresql.org, so you can use CVS to update the +main source tree, or you can ftp your files into your account, patch, +and cvs install the changes directly into the source tree. <P> + +<H3><a name="6">6</a>) How do I test my changes?</H3><P> + +First, use <I>psql</I> to make sure it is working as you expect. Then +run <I>src/test/regress</I> and get the output of +<I>src/test/regress/checkresults</I> with and without your changes, to +see that your patch does not change the regression test in unexpected +ways. This practice has saved me many times. The regression tests test +the code in ways I would never do, and has caught many bugs in my +patches. By finding the problems now, you save yourself a lot of +debugging later when things are broken, and you can't figure out when it +happened.<P> + + +<H3><a name="7">7</a>) I just added a field to a structure. What else +should I do?</H3><P> + +The structures passing around from the parser, rewrite, optimizer, and +executor require quite a bit of support. Most structures have support +routines in <I>src/backend/nodes</I> used to create, copy, read, and output +those structures. Make sure you add support for your new field to these +files. Find any other places the structure may need code for your new +field. <I>mkid</I> is helpful with this (see above).<P> + + +<H3><a name="8">8</a>) Why are table, column, type, function, view +names sometimes referenced as <I>Name</I> or <I>NameData,</I> and +sometimes as <I>char *?</I></H3><P> + +Table, column, type, function, and view names are stored in system +tables in columns of type <I>Name.</I> Name is a fixed-length, +null-terminated type of <I>NAMEDATALEN</I> bytes. (The default value +for NAMEDATALEN is 32 bytes.) + +<PRE><CODE> + typedef struct nameData + { + char data[NAMEDATALEN]; + } NameData; + typedef NameData *Name; +</CODE></PRE> + +Table, column, type, function, and view names that come into the +backend via user queries are stored as variable-length, null-terminated +character strings.<P> + +Many functions are called with both types of names, ie. <I>heap_open().</I> +Because the Name type is null-terminated, it is safe to pass it to a +function expecting a char *. Because there are many cases where on-disk +names(Name) are compared to user-supplied names(char *), there are many +cases where Name and char * are used interchangeably.<P> + +<H3><a name="9">9</a>) How do I efficiently access information in +tables from the backend code?</H3><P> + +You first need to find the tuples(rows) you are interested in. There +are two ways. First, <I>SearchSysCacheTuple()</I> and related functions +allow you to query the system catalogs. This is the preferred way to +access system tables, because the first call to the cache loads the +needed rows, and future requests can return the results without +accessing the base table. The caches use system table indexes +to look up tuples. A list of available caches is located in +<I>src/backend/utils/cache/syscache.c.</I> +<I>src/backend/utils/cache/lsyscache.c</I> contains many column-specific +cache lookup functions.<P> + +The rows returned are cached-owned versions of the heap rows. They are +invalidated when the base table changes. Because the cache is local to +each backend, you may use the pointer returned from the cache for short +periods without making a copy of the tuple. If you send the pointer +into a large function that will be doing its own cache lookups, it is +possible the cache entry may be flushed, so you should use +<I>SearchSysCacheTupleCopy()</I> in these cases, and <I>pfree()</I> the +tuple when you are done.<P> + +If you can't use the system cache, you will need to retrieve the data +directly from the heap table, using the buffer cache that is shared by +all backends. The backend automatically takes care of loading the rows +into the buffer cache.<P> + +Open the table with <I>heap_open().</I> You can then start a table scan +with <I>heap_beginscan(),</I> then use <I>heap_getnext()</I> and +continue as long as <I>HeapTupleIsValid()</I> returns true. Then do a +<I>heap_endscan().</I> <I>Keys</I> can be assigned to the <I>scan.</I> +No indexes are used, so all rows are going to be compared to the keys, +and only the valid rows returned.<P> + +You can also use <I>heap_fetch()</I> to fetch rows by block +number/offset. While scans automatically lock/unlock rows from the +buffer cache, with <I>heap_fetch(),</I> you must pass a <I>Buffer</I> +pointer, and <I>ReleaseBuffer()</I> it when completed. + +Once you have the row, you can get data that is common to all tuples, +like <I>t_self</I> and <I>t_oid,</I> by merely accessing the +<I>HeapTuple</I> structure entries. + +If you need a table-specific column, you should take the HeapTuple +pointer, and use the <I>GETSTRUCT()</I> macro to access the +table-specific start of the tuple. You then cast the pointer as a +<I>Form_pg_proc</I> pointer if you are accessing the pg_proc table, or +<I>Form_pg_type</I> if you are accessing pg_type. You can then access +the columns by using a structure pointer: + +<PRE> +<CODE> + ((Form_pg_class) GETSTRUCT(tuple))->relnatts +</CODE> +</PRE> + +You should not directly change <I>live</I> tuples in this way. The best +way is to use <I>heap_tuplemodify()</I> and pass it your palloc'ed +tuple, and the values you want changed. It returns another palloc'ed +tuple, which you pass to <I>heap_replace().</I> + +You can delete tuples by passing the tuple's <I>t_self</I> to +<I>heap_destroy().</I> You can use it for <I>heap_update()</I> too. + +Remember, tuples can be either system cache versions, which may go away +soon after you get them, buffer cache versions, which go away when +you <I>heap_getnext(),</I> <I>heap_endscan,</I> or +<I>ReleaseBuffer()</I>, in the <I>heap_fetch()</I> case. Or it may be a +palloc'ed tuple, that you must <I>pfree()</I> when finished. + +<H3><a name="10">10</a>) What is elog()?</H3><P> + +<I>elog()</I> is used to send messages to the front-end, and optionally +terminate the current query being processed. The first parameter is an +elog level of <I>NOTICE,</I> <I>DEBUG,</I> <I>ERROR,</I> or +<I>FATAL.</I> + +<I>NOTICE</I> prints on the user's terminal and the postmaster logs. +<I>DEBUG</I> prints only in the postmaster logs. <I>ERROR</I> prints in +both places, and terminates the current query, never returning from the call. +<I>FATAL</I> terminates the backend process. + +The remaining parameters of <I>elog</I> are a <I>printf</I>-style set of +parameters to print. + +<H3><a name="11">11</a>) What is configure all about?</H3><P> + +The files <I>configure</I> and <I>configure.in</I> are part of the +GNU <I>autoconf</I> package. Configure allows us to test for various +capabilities of the OS, and to set variables that can then be tested in +C programs and Makefiles. Autoconf is installed on the PostgreSQL main +server. To add options to configure, edit <I>configure.in,</I> and then +run <I>autoconf</I> to generate <I>configure.</I><P> + +When <I>configure</I> is run by the user, it tests various OS +capabilities, stores those in <I>config.status</I> and +<I>config.cache,</I> and modifies a list of <I>*.in</I> files. For +example, if there exists a <I>Makefile.in,</I> configure generates a +<I>Makefile</I> that contains substitutions for all @var@ parameters +found by configure.<P> + +When you need to edit files, make sure you don't waste time modifying +files generated by <I>configure.</I> Edit the <I>*.in</I> file, and +re-run <I>configure</I> to recreate the needed file. If you run <I>make +distclean</I> from the top-level source directory, all files derived by +configure are removed, so you see only the file contained in the source +distribution.<P> + +<H3><a name="12">12</a>) How do I add a new port?</H3><P> + +There are a variety of places that need to be modified to add a new +port. First, start in the <I>src/template</I> directory. Add an +appropriate entry for your OS. Also, use <I>src/config.guess</I> to add +your OS to <I>src/template/.similar.</I> You shouldn't match the OS +version exactly. The <I>configure</I> test will look for an exact OS +version number, and if not found, find a match without version number. +Edit <I>src/configure.in</I> to add your new OS. (See configure item +above.) You will need to run autoconf, or patch <I>src/configure</I> +too.<P> + +Then, check <I>src/include/port</I> and add your new OS file, with +appropriate values. Hopefully, there is already locking code in +<I>src/include/storage/s_lock.h</I> for your CPU. There is also a +<I>src/makefiles</I> directory for port-specific Makefile handling. +There is a <I>backend/port</I> directory if you need special files for +your OS.<P> + + +</BODY> +</HTML> |