Logging in GnuCash

GnuCash leverages glib logging, with some historical and convenience macros. As of Winter 2007, GnuCash has an ad-hoc layer on top of glib's "gmessage.h" logging inspired by log4j. In short, messages are logged in a given log-domain and at a specific level. The log-domain is a "."-separated path, and a table of log levels is configured. Between the two, a tree of log-level is established. Log messages are checked against the tree before being emitted.

[edit] log.conf

The special file ~/.gnucash/log.conf is checked for at gnucash start up; if present, its contents will configure the logging subsystem. The primary purpose of the configuration file it to setup a specific set of log-levels, though it's also useful for changing the default logging location from /tmp/gnucash.trace to stdout, stderr or another file.

The specific set of log paths that exist are going to be a function of the code at any given point, but some basic patterns will stabilize. As of Feb 13 2007, User:Jsled is using the following to generally raise the log level of the app, but also to get debugging detail about file-backend and U I code changes related to scheduled transactions:

   [levels]
   gnc=message
   
   gnc.bin=info
   gnc.gui=info
   gnc.backend.file=info
   
   gnc.engine.sx=debug
   gnc.app-util.sx=debug
   gnc.backend.file.sx=debug
   gnc.gui.sx=debug
   gnc.gui.frequency=debug
   
   GLib=error
   
   [output]
   to=stderr

Note that these settings are probably not suitable for your purpose, but serve as an example of what's possible.

Git

git-svn clone -r16500:HEAD svn+ssh://USERNAME@svn.gnucash.org/repo/gnucash/trunk

git-svn rebase

git-svn dcommit

git-svn clone -s -r16500:HEAD svn+ssh://USERNAME@svn.gnucash.org/repo/gnucash all
cd all
git branch --track my-trunk remotes/trunk
git branch --track my-2.2 remotes/2.2
cd ..

cd all
git checkout my-trunk
cd ..
git-clone -s all trunk

cd all
git checkout my-2.2
cd ..
git-clone -s all 2.2

cd 2.2
./autogen.sh
mkdir build
cd build
../configure --some-options-foo-bar --prefix=/opt/experimental .....

git-svn clone -s -r16500:HEAD svn+ssh://USERNAME@svn.gnucash.org/repo/gnucash
git reset --hard remotes/trunk

Quicken Interchange Format (QIF) is an open specification for reading and writing financial data to media (i.e. files). A QIF file typically has the following structure:

!Type:type identifier string
[single character line code]Literal String Data
...
^
[single character line code]Literal String Data
...
^


Each record is ended with a ^ (caret). All the data in the file is stored in ASCII strings, and the file could be edited in any text editor

QIF is older than Open Financial Exchange (OFX). The inability to reconcile imported transactions against the current account information is one of the primary shortcomings of QIF. It is commonly supported by financial institutions to supply downloadable information to account holders. Most personal money management software (Microsoft Money, Intuit's Quicken versions earlier than 2006) can read QIF files to import information. However, Quicken has dropped support of QIF for online banking accounts as of version 2006 [1]. However, as of release R4, Quicken 2006 still supports importing and exporting transactions in QIF format for asset, liability, cash, small business payable, and invoice accounts [2]. Banks that support integrated online banking (i.e. as part of Money or Quicken) usually use OFX instead of QIF.

Note: If you really need to get data from a QIF file into an account unsupported for QIF import, you can can import from the QIF file into a (temporary) Cash account, then in the (temporary) account register select all the transactions, right click on them and select Cut Transaction(s), goto the register you want the items to be in, then right click in the account register and select Paste Transaction(s). Tested in 2006 version.

Some banks are objecting to this change, because Quicken (Intuit) charges licensing fees to use OFX. Other banks are passing on these fees, by charging customers for downloading OFX files. (QIF is an open format, free for anyone to use. Banks do not normally charge for downloading QIF files.) Microsoft Money imports either QIF or OFX format files, and Microsoft does not charge banks any licensing fees to use OFX for Money.

Required File Formatting

Each transaction must end with a symbol, indicating the end of entry. Each item in the transaction must display on a separate line. When Quicken exports an account register or list, it adds a line to the top of the file that identifies the type of account or list. Listed below are the header lines Quicken adds to the exported files:

Quicken can be configured to import all transfers, regardless of whether Ignore Transfers is selected when the file is imported. To do this, add a line to the file being imported into the Quicken account. Use a text editor or word processor to put the following line immediately after the header line at the top of the file:

!Option:AllXfr

Items for Non-Investment Accounts

Each item in a bank, cash, credit card, other liability, or other asset account must begin with a letter that indicates the field in the Quicken
register. The non-split items can be in any sequence:

Note: Repeat the S, E, and $ lines as many times as needed for additional items in a split. If an item is omitted from the transaction in the QIF file, Quicken treats it as a blank item.

Items for Investment Accounts

Items for Account Information

The account header !Account is used in two places, at the start of an account list and the start of a list of transactions to specify to which account they belong.

Items for a Category List

Items for a Class List

Items for a Memorized Transaction List

Immediately preceding the ^ character, each entry must end with one of the following file indicators to specify the transaction type.

KC

KD

KP

KI

KE

With that exception, memorized transaction entries have the same format as regular transaction entries (non-investment accounts). However, the Date or Num field is included. All items are optional, but if an amortization record is included, all seven amortization lines must also be included.

Examples of QIF files

Normal Transactions Example

Investment Example

Memorized List Example

Generic (Import) Druid Framework

Work in progress

Background

If we were to make the importer backend-driven instead of GUI-driven we would necessarily require nested gtk_main()'s. The reason is that the backend was originally executed from GUI callback, so what should happen is that the initialization sets up the import process and then returns control so we dont have a nested gtk_main().

Nesting gtk_main() can result in crashes. If the originating window is destroyed then when control returns we've jumped back into invalid data space. This has happened on numerous occasions within the gnucash code over time and has caused numerous bugs. For example, the 'Save while saving crashes gnucash' bug was due to a nested gtk_main caused by the progress-bar. Consequently, the best way to avoid this problem is not to introduce this problem and avoid nested gtk_main() whenever possible. This means the importer should be GUI-driven, not backend-driven.

The Problem

Another issue is that some sub-processes of the importing process require multiple druid pages. If this sub-process is repeatable, it means the druid needs to be able to jump back to the beginning of the sub-process. For example, the process to choose files to import should allow users to import multiple files at one time.

Moreover, even when a backend may sometimes require access to particular druid sub-process, it may need to skip that sub-process sometimes. For example, the QIF importer may have an ambiguity in the date format for a file, requiring the user to choose the actual date format. However if the imported file is not ambiguous this sub-process of the druid can be skipped.

All of this means the druid framework should be able to rotate across a subset of the pages for a sub-process or skip pages for a sub-process based on the requirements of the backend.

In addition the framework should allow a global setting to enable or disable "documentation pages" (c.f. the Show QIF Documentation preference). Each sub-process can have a set of doc pages available which can be displayed (or not) based on a user preference.

The Druid Framework

Each provider implements the Provider API and Provider Registrar API and registers itself with the druid provider registry. The Provider Registrar API defines the minimal set of functions and methods to instantiate a provider and place a set of one or more pages into a druid instance. The Provider API is used to set up the callbacks to hook that provider into the backend.

In addition to the standard Provider API, each provider must define a private API for use with the backend. Each provider is going to interact with the backend differently, so there is no way to define a common API for this interaction. On the other hand, the backend already knows a priori which providers it needs to use, so it can know the provider-dependent API and use that interaction.

The druid provider registry allows the Druid Builder to combine the providers in the requested order when a backend asks to build a druid. It uses the Provider Registrar API to instantiate a Provider and hook it into the Druid, and then uses the Provider API to connect the Provider to the backend using the data provided by the backend.

This leaves the importer backend blissfully unaware of the actual Druid GUI/toolkit implementation (i.e. it doesn't need to know that the Druid is actually a GnomeDruid -- it could be some other UI toolkit). The backend calls the Druid Builder to put together the druid with the appropriate providers and supplies the provider-specific callback information necessary to hook into the druid.

In order to initiate an import for a particular backend, the GUI calls into the "start import" routine which builds an import context, builds the import druid, and then returns control let the GUI run. This means the import backend should be completely callback-based, including the cleanup code in case the user interrupts the import.

Druid Sub-Process Providers

The interface between the Provider and the Builder is obviously toolkit-specific because the builder need to piece together the actual druid pages (e.g. GnomeDruidPageStandard). The druid builder requests an instance of a provider (and its pages) and supplies the provider with the backend callbacks requests. The provider instance creates the druid pages and connects the passed-in callback data so it can call the backend appropriately.

Each provider necessarily requires its own callback interface (because each provider needs to supply different data to the backend in different ways). This is implemented by subclassing the basic callback storage type. Because the importer backend knows the providers being used, it can provide the required callback storage type when it builds the druid.

When a user fills in a druid page and clicks on "Next" the druid will call the next-page callback and supply the provided data (as defined in the particular provider callback API). The backend then acts on the callback data, sets the next page in the druid, and returns control. Similar operations occur when the user clicks "Back", a back-page callback, or any other callbacks required by the specific provider.

Putting the Druid Together

Linking into the Backend

Core GncDruid Types

GncDruidPage

GncDruidCB

Members:

gpointer backend_ctx;

GncDruid

Members:

void set_page(
GncDruid,
GncDruidPage);

Set the current page of the druid to the GncDruidPage.

GncDruidProvider current_provider;
GncDruidProvider (*next_provider)(GncDruid);
GncDruidProvider (*prev_provider)(GncDruid);

GncDruidProviderDesc

Members:

const gchar *name;
gboolean (*provider_needed)(GncDruidCB);
gboolean (*next_cb)(GncDruid, GncDruidCB);
gboolean (*prev_cb)(GncDruid, GncDruidCB);

GncDruidProvider

Members:

GncDruidPage (*first_page)(GncDruidProvider);
GncDruidPage (*next_page)(GncDruidProvider);
GncDruidPage (*prev_page)(GncDruidProvider);

The Druid Builder API

Build a druid using the supplied list of providers descriptions (GncDruidProviderDesc). The provider list also contains all the callback information necessary to hook the backend into the druid. The backend_ctx and end() parameter are used to end the session in the case of the user clicking "cancel". It cleans up the backend context.

The Basic (internal) Provider APIs

Obtain an instance of a Druid Provider based on the Provider Description. This is used by the druid builder to obtain a Provider Instance given the Provider Description.

void gnc_provider_register(
const gchar* name,
gnc_provider_get_instance);

Register a Provider Implementation of the provided name. Provide a creation function that gnc_provider_get_instance() can use to obtain a fully initialized provider object.

Specific GncDruidProviderDesc APIs

GncDruidProviderDescChooseFmt

A provider that allows the user to choose a format in the case of ambiguous input.

Members:

void (*get_ambiguity)(
gpointer be_ctx,
GncImportFormat* choices,
GncImportFormat* last_choice);

GncDruidChooseFmtCB

Members:

GncImportFormat choice;

GncDruidProviderDescSelectFile

A provider that allows the user to select the file(s) to import.

Members:

gboolean multi_file;
gchar* last_directory;
GList * (*get_files)(gpointer be_ctx);
const gchar* (*get_filename)(GncImportFile file);
void (*remove_file)(gpointer be_ctx, GncImportFile file)

GncDruidSelectFileCB

Members:

gchar* filename;