Command line clients

From refbase

This page documents the refbase command line clients that come with refbase >v0.8.0.

1 'refbase' command line client
2 'refbase_import' command line client

'refbase' command line client

The 'refbase' command line client allows you to search a refbase online database from the command line and retrieve results in various formats. By default, found results are returned as citations in plain text format, but search results can easily be saved as HTML, RTF, PDF, LaTeX or structured text (Markdown). In addition, found results can be exported to BibTeX, Endnote, RIS or XML (MODS, SRW, OpenOffice.org/OpenDocument) formats.

Besides using the refbase CLI to perform a quick database query from the shell, the command line client allows for easy integration of a refbase database into a custom workflow. Possible applications could be automatic backups via scheduled cron jobs or automatic creation and update of static publication lists that employ a custom HTML layout.

In addition, the 'refbase' command line client enables you to append found records to an offline BibTeX or MODS/SRW XML file (if they don't yet exist in that file), and update existing records in that file if their modification date on the server is more recent. This allows you to keep an offline file in sync with any additions or changes made in the refbase online database.

[edit]

Requirements

The 'refbase' command line client requires:

a shell with Perl execution capabilities
the Perl CPAN modules LWP::UserAgent, HTTP::Request::Common, HTTP::Response, HTTP::Cookies (all part of the libwww-perl package) and URI::URL
access to a refbase database (refbase-0.9.0 or greater, refbase >v0.9.0 required for --append and --update mode)

[edit]

Installation

Install required Perl modules (if missing)

Info about installing Perl modules from CPAN is available here and here. Here's a quick guide: you can check if the above mentioned CPAN modules are already installed on your system using these commands from your shell:

perl -MLWP::UserAgent -e 1
perl -MHTTP::Request::Common -e 1
perl -MHTTP::Response -e 1
perl -MHTTP::Cookies -e 1
perl -MURI::URL -e 1

If these commands don't return any error messages, you have the required modules. If you do see some error messages, it's still possible that these modules are installed on your system, but that they are not in your path, which can be displayed with:

perl -e "print qq(@INC)"

If the required CPAN modules are not installed on your system, you should be able to install them using the following commands:

perl -MCPAN -e 'install LWP::UserAgent'
perl -MCPAN -e 'install HTTP::Request::Common'
perl -MCPAN -e 'install HTTP::Response'
perl -MCPAN -e 'install HTTP::Cookies'
perl -MCPAN -e 'install URI::URL'

The installation process will try to take care of any dependencies. However, if your module installation fails because of other missing modules, checkout the error messages and try to install the mentioned modules first, then try again to install the above mentioned modules.

Download 'refbase' CLI

Download the latest script version of the refbase command line client from the SourceForge SVN as source text.

Copy 'refbase' CLI to executable path

Put the downloaded script file within an executable path and make sure that the file itself is executable.

You should now be able to type refbase at the command line to invoke the 'refbase' command line client. See below for more info about the required syntax.

[edit]

Syntax

For help with the syntax type:

refbase -h
refbase --help

This will print the usage screen:

refbase command line client, v1.2 by Matthias Steffens, http://cli.refbase.net/

Usage:   refbase [OPTIONS]

Notes:   - At least one query option must be given and unrecognized options will be ignored.
         - If multiple options are given, they will by default be connected with 'AND'. Use
           '--query=or' to connect multiple options with 'OR'.
         - Options syntax: [OPTION]=[VALUE], e.g. '-a=steffens' or '--author="steffens, m"'.
         - Returns up to '--rows' number of records beginning with '--start'. If all given
           query options are empty, all database records will be returned.
         - Note that '--records' assumes a list of full record serials separated by non-digit
           characters while '--serial' allows for partial matches.
         - For each option, default values can be specified at the top of the script.
           Current defaults are given in parentheses.

General Options:   -h, --help        - display this help text
                   -v, --version     - display version information
                   -X, --examples    - display usage examples

Query Options:     -a, --author      - search author field (' ')
                   -b, --abstract    - search abstract field (' ')
                   -c, --citekey     - search cite_key field, requires '-u, --userid' (' ')
                   -d, --date        - search by creation date (' ')
                   -e, --area        - search area field (' ')
                   -f, --thesis      - search thesis field (' ')
                   -i, --contribid   - search contribution_id field (' ')
                   -j, --journal     - search abbrev_journal field (' ')
                   -k, --keywords    - search keywords field (' ')
                   -l, --location    - search location field (' ')
                   -m, --marked      - search marked field, requires '-u, --userid' (' ')
                   -n, --notes       - search notes field (' ')
                   -p, --publication - search publication field (' ')
                   -q, --query       - query type, possible values: and, or ('and')
                   -r, --records     - search serial field (' ')
                   -s, --selected    - search selected field, requires '-u, --userid' (' ')
                   -t, --title       - search title field (' ')
                   -u, --userid      - join with user-specific data from user ID (' ')
                   -w, --where       - search by using a raw sql where clause (' ')
                   -x, --type        - search type field (' ')
                   -y, --year        - search year field (' ')
                   -z, --serial      - search serial field (partial matches) ('.+')

Output Options:    -A, --append      - file to which returned records are appended (' ')
                                       requires '-F, --format': bibtex, mods, srw
                   -B, --update      - update existing records in '-A, --append' file ('0')
                                       possible values: 0, 1
                   -C, --style       - citation style (' ')
                   -E, --extract     - file from which citation IDs are extracted (' ')
                                       supported file types: aux, bbl, bib, default, enw, ris, tex, xml
                   -F, --format      - output format ('ascii')
                                       possible values: html, rtf, pdf, latex, latex_bbl, markdown, ascii,
                                                        bibtex, endnote, ris, isi, mods, srw, odf, word
                   -L, --showlinks   - hide/display links column in html output ('1')
                                       possible values: 0, 1
                   -O, --order       - sort order of returned records ('author')
                                       possible values: author, year, type, type-year
                   -Q, --showquery   - hide/display SQL query in ASCII output ('0')
                                       possible values: 0, 1
                   -R, --rows        - number of records to be returned (' ')
                   -S, --start       - number of first record to be returned ('1')
                   -V, --view        - view type of html output ('web')
                                       possible values: web, print

Server Options:    -H, --host        - URL of the refbase database ('https://refbase.ipoe.uni-kiel.de/refs/')
                                       defined shortcuts: beta, default, demo, local, org
                   -P, --password    - password for given '-U, --user' account (' ')
                   -U, --user        - login email address of an existing refbase user
                                       (' ')

[edit]

Examples

To view some usage examples type:

refbase -X
refbase --examples

This will print the examples screen:

--------------------------------------------------------------------------------
REFBASE USAGE EXAMPLES:
--------------------------------------------------------------------------------

1) Find all records where the author field contains 'mock' AND the year field
   contains '2005':

   refbase -a=mock -y=2005

--------------------------------------------------------------------------------

2) Find all records where the author field contains 'mock' OR the title field
   contains 'photo', and display 10 records starting with the 21st record in the
   result set:

   refbase -a=mock -t=photo -q=or -R=10 -S=21

--------------------------------------------------------------------------------

3) Export records with serial numbers '1', '12' and '34' to Endnote format and
   save them to a file named 'export.enw':

   refbase -r=1,12,34 -F=endnote > export.enw

--------------------------------------------------------------------------------

4) Return up to 50 records that were selected by a user with a user ID '2' in
   RTF format using citation style "Ann Glaciol" and sorting them first by
   record type, then by year, and save results to a file named 'citations.rtf':

   refbase -s=yes -u=2 -R=50 -F=rtf -C="Ann Glaciol" -O=type-year > citations.rtf

--------------------------------------------------------------------------------

5) Find all records which were modified today by a user named "admin" and where
   the location field contains 'msteffens' (note the use of the '-w' option to
   specify a custom WHERE clause):

   refbase -w='modified_date = CURDATE() AND modified_by RLIKE "admin"' -l=msteffens

--------------------------------------------------------------------------------

6) Find all records where the cite_key field (of a user with a user ID '2')
   contains 'steffens', and append records in MODS XML format to file 'mods.xml'
   if they don't yet exist in that file:

   refbase -u=2 -c=steffens -F=mods -A=mods.xml

--------------------------------------------------------------------------------

7) Find all records where the contribution_id field contains 'AWI' and where
   the keywords field contains 'seaweeds', and append records in BibTeX format
   to file 'paper.bib' if they don't yet exist in that file. In case found
   records already exist in file 'paper.bib', update them if their modification
   date is more recent:

   refbase -i=AWI -k=seaweeds -F=bibtex -A=paper.bib -B=1

--------------------------------------------------------------------------------

8) Extract all citation IDs from file 'paper.aux', and append matching records
   (for a user with a user ID '2') in BibTeX format to file 'paper.bib' if they
   don't yet exist in that file. In case found records already exist in file
   'paper.bib', update them if their modification date is more recent:

   refbase -u=2 -E=paper.aux -F=bibtex -A=paper.bib -B=1

--------------------------------------------------------------------------------

9) Extract all citation IDs from file 'bibtex.bbl', and save matching records
   (for a user with a user ID '2') in LaTeX bibliography (.bbl) format to file
   'refbase.bbl' using the "APA" citation style (the .bbl file generated by
   refbase can be used as a replacement of the BibTeX-generated .bbl file):

   refbase -u=2 -E=bibtex.bbl -F=latex_bbl -C=APA > refbase.bbl

--------------------------------------------------------------------------------

[edit]

Appending to a file (export all records)

PHP has memory and time-execution limits, which prevent VERY large export sets. These can pose a hurdle for those who wish to export a substantial portion of their database (for example to share with Endnote or BibTeX using colleagues). The following shell script will export all references which have serial numbers between 1 and 10,000 (one-by-one, appended to the same file):

for ((i=1; i<=10000;i++))
do
  ./refbase -r=$i -F=bibtex >> output.bib
done

The script may be trivially modified with a different range of serial numbers or to export to a different tagged format (such as "endnote").

You may also want to modify it to request more than one record at once and/or to wait between requests.

[edit]

Current limitations

Currently, this utility supports search & retrieve, but does not support update actions such as add, edit or delete.

This script is currently just an interface to show.php, which for example does not support arbitrary sort orders.

Specifying the record offset (--start) as well as the number of records to be returned (--rows) will only work for the formats html, rtf, pdf, latex, latex_bbl, markdown, ascii and srw, since the other formats are designed to always export the entire result set. Note that for html, --start is adjusted to the next lower value that is an exact multiple of --rows (which ensures correct behaviour of the browse links).

The authentication mechanism is currently limited in that a given password will be transferred as parameter in the POST request.

The --append and --update modes currently only work with formats bibtex, mods and srw.

[edit]

'refbase_import' command line client

The 'refbase_import' command line client enables you to upload data in various formats to a refbase online database from the command line. This allows for easy integration of a refbase database into a custom workflow. A possible application could be the automatic retrieval & import of newly added records from PubMed via a scheduled cron job.

[edit]

Requirements

The 'refbase_import' command line client requires:

a shell with Perl execution capabilities
the Perl CPAN modules LWP::UserAgent, HTTP::Request::Common, HTTP::Response, HTTP::Cookies (all part of the libwww-perl package) and URI::URL
access to an refbase-0.9.0 or above

[edit]

Installation

Install required Perl modules (if missing)

See the 'refbase' command line client for a detailed description on how to install any required Perl modules.

Download 'refbase_import' CLI

Download the latest script version of the 'refbase_import' command line client from the SourceForge SVN as source text.

Copy 'refbase_import' CLI to executable path

Put the downloaded script file within an executable path and make sure that the file itself is executable.

You should now be able to type refbase_import at the command line to invoke the 'refbase_import' command line client. See below for more info about the required syntax.

[edit]

Syntax

For help with the syntax type:

refbase_import -h
refbase_import --help

This will print the usage screen:

refbase_import command line client, v1.0 by Matthias Steffens, http://cli.refbase.net/

Usage:   refbase_import [OPTIONS] [FILE]

Notes:   - Two import modes are supported:
           1) '--type=data' requires an import FILE to be specified;
              for supported import formats, see: http://import.refbase.net/
           2) '--type=pmid' requires the '-p|--pmid' option with one or more PubMed IDs
              delimited by any non-digit characters.
         - Options syntax: [OPTION]=[VALUE], e.g. '-p=16351846' or '--pmid="16351846 16783713"'.
         - For each option, default values can be specified at the top of the script.
           Current defaults are given in parentheses.

General Options:   -h, --help       - display this help text
                   -v, --version    - display version information
                   -X, --examples   - display usage examples

Import Options:    -b, --skipbad    - skip records with unrecognized data format ('0')
                                      possible values: 0, 1
                   -i, --import     - import all or only some records ('all')
                                      possible values: all, only
                   -p, --pmid       - PubMed IDs of records to import (' ')
                   -r, --records    - positional numbers and/or ranges of records to import ('1')
                                      requires the '--import=only' option
                   -t, --type       - import type ('data')
                                      possible values: data, pmid

Server Options:    -H, --host       - URL of the refbase database ('http://beta.refbase.net/')
                                      defined shortcuts: default, local
                   -P, --password   - password for given user account (' ')
                   -U, --user       - login email address of an existing refbase user with
                                      import permissions (' ')

[edit]

Examples

To view some usage examples type:

refbase_import -X
refbase_import --examples

This will print the examples screen:

 --------------------------------------------------------------------------------
 REFBASE_IMPORT USAGE EXAMPLES:
 --------------------------------------------------------------------------------

 1) Import BibTeX records from file 'import.bib' using the defaults defined
    within the refbase_import script:
 
    refbase_import import.bib
 
 --------------------------------------------------------------------------------

 2) Import all Endnote records given in file 'import.enw' into your default
    refbase database:
 
    refbase_import -t=data -i=all import.enw
 
 --------------------------------------------------------------------------------

 3) Take RIS records from file 'import.ris' but import only the first three as
    well as the fifth and the tenth record into your local refbase database:
 
    refbase_import -H=local -t=data -i=only -r=1-3,5,10 import.ris
 
 --------------------------------------------------------------------------------

 4) Fetch two records from PubMed via their PMID identifier (in this example,
    records with PMIDs 16351846 and 16783713) and import them into your local
    refbase database:
 
    refbase_import -H=local -t=pmid -p=16351846,16783713
 
 --------------------------------------------------------------------------------

[edit]

Current limitations

The character encoding of your import data must match the encoding of your refbase database (i.e., 'latin1' or 'utf8').

The authentication mechanism is currently limited in that a given password will be transferred as parameter in the POST request.

Retrieved from "http://www.refbase.net/index.php/Command_line_clients"

Categories: Usage

Command line clients

From refbase

Contents

'refbase' command line client

Requirements

Installation

Syntax

Examples

Appending to a file (export all records)

Current limitations

'refbase_import' command line client

Requirements

Installation

Syntax

Examples

Current limitations

Views

Personal tools

Search

Toolbox