Hackathon 2013/Citations

This pitch covers making sure that citations can flow easily in and out of TaxonWorks. It will do this in three ways:
 * 1) Finding an existing data model which can hold information citations with a standard format for representing it.
 * 2) * Possibly BibTeX
 * 3) * Possibly http://bibliontology.com/
 * 4) Writing code to read this in and out in Ruby, possibly just using a standard library
 * 5) * Possibly http://rubygems.org/gems/bibtex-ruby
 * 6) Writing code to resolve microcitations. A working example of this has been created by Rod Page that links generic names from Nomenclator Zoologicus with references from the Biodiversity Heritage Library.
 * 7) * Micro-citations should be differentiated from verbatim references. A micro-citation is just an author and date (as seen in a full taxon name). A verbatim reference is the full reference as it appears in documentation that hasn't been broken into normalized pieces yet. Both versions need to be tracked and be searchable. We also need a way to get a listing out of TW of both versions, so they can be normalized into full sources.
 * 8) * Lists of journals.
 * 9) * BHL API to convert BHL URLs into references
 * 10) Finding out more about a particular citation
 * 11) * Getting abstract, keywords from PubMed
 * 12) * Pulling in citation from:
 * 13) ** BHL
 * 14) ** ZooRecord
 * 15) ** Biosys
 * 16) * ImpactStory information
 * 17) Automatically parsing citations into authors, title, etc.
 * 18) * Lots of cool stuff at http://biblio.globalnames.org/
 * 19) Designing a user interface to make it easy to resolve microcitations
 * 20) * Autocompletion
 * 21) * Journal name identification
 * 22) * Searching on Google Scholar/Wikipedia for books/authors

Members

 * Beth Frank
 * Gaurav Vaidya
 * Mike Maehr
 * Marilyn Beckman
 * Dmitry Dmitriev

Requirements (potential test cases/use cases)

 * Must support letters after the year for multiple publications of the same author in the same year.
 * No single field will be required for any given record - this is different than the requirements for valid BibTeX.
 * The scientist should be able to enter partial or incomplete data, then return later to complete this information. TW must support a workflow that is convenient for the scientist, not the program. (That may be all that is conveniently available at the moment.) (See note below on adding a "review needed" tag on import. This type of tag should be available on manual entry as well.)
 * The scientist should be able to copy a verbatim reference from another document and pass it into TW for normalization later.
 * The scientist should be able to similarly pass just a URL/URN or other identifier in as a reference for completion later (this will be a source of type "miscellaneous"). Ideally this could then simply resolve identifiers such as a DOI and generate readily available reference information.
 * When importing SF reference data into TW sources, they will need to be tagged "For review" because the BibTex has a finer grain of differentiation of types than SF.
 * Should be able to store abstracts, nomenclature acts & entire classification (available from ZooRecord - most probably returned as text strings).
 * Should be able to round trip data (e.g. import a BibTex file, then output a BibTex file and have them be the same.)

Coding Notes
The following are notes retrieved from Matt's VUE ([[Media:Source.JPG]]) file relating to Sources.


 * Relationships to other objects within TaxonWorks:
 * Sources may have a SourceAuthor and/or SourceEditor (Role of a person)
 * Sources may have a HumanSource (Requires a person with a role of SourceSource)
 * Sources will support the following BibTex types:
 * Book
 * Article - an article is published in a Serial (Journal)
 * Conference
 * Booklet
 * InBook
 * InCollection
 * MastersThesis
 * InProceedtings
 * Misc - this will be used when the only available current source is a URL.
 * PhdThesis
 * Techreport
 * Unpublished - TaxonWorks revisions and other works-in-progress, may also be used for LepIndex catalogue cards.
 * Manual
 * Sources may be published in a Serial, which will support relationships of Preceding and Succeeding.
 * Serials have:
 * Title
 * Series_year_start
 * Series_year_end
 * editors (text list - different from people with roles)
 * publisher
 * place_published
 * primary_language
 * SerialRelationship is modeled as type & 2 serial IDs
 * SerialRelationshipType will be based on MARC (http://www.oclc.org/bibformats/en/7xx.html - see 780 and 785)

Open Issues

 * How do we support Tom in Tom, Dick & Harry (E.G. Tom is the authority but the actual journal article is by Tom, Dick & Harry). Are these separate sources (Dmitry says no - not supporting ref-in-ref the same way that SF does. In this case, the taxonomic authority string (which is just a text string) would just not match the author string in the original description source). Ed is wondering, do we divorce the authors of taxa from authors in references? Otherwise how, would we associate the authors for a species correctly.


 * Would like a way to store linkages between sources and OTUs that indicate the type of information within the source (e.g. key, images). SF has tblCiteInfoFlags.FlagValue which specifies this kind of information. It is also summarized across all citations to the same reference in tblReferences.CiteDataStatus.

Deliverables

 * 1) Use bibtex-ruby to read, write and round-trip bibliographic information.
 * 2) Write a Rails system for storing citations, and integrate it with bibtex-ruby.
 * 3) * Beth
 * 4) Given an identifier, look for information about it online.
 * 5) * Gaurav
 * 6) Parsing citations in Ruby.

Terms

 * Citation: An individual, unnormalized use of a source.
 * Each citation will have a unique identifier that the rest of the system can reference.
 * It must be possible to have a citation which consists ONLY of a single identifier. We could treat this as a verbatim reference.
 * Source: Someone or institution credited as providing data.
 * TW needs sources to be private or public. Why?
 * Global source: a common pool of sources. These should be published and non-private.

Datasets we can play with
How do these fit here? - These are online resources that we can pull datasets from (Beth)
 * ITIS
 * GNUB
 * UCD

Input/output formats

 * BibTeX
 * RIS?
 * Mark21 (Library of Congress Standard)
 * BibJSON
 * RDF/XML

URLs and identifiers of taxonomic significance
It should be noted that there will be multiple identifiers associated with a single source.
 * ISBN/ISSN
 * BHL URLs
 * PubMed ID/URLs
 * DOI ID/URLs
 * Handle ID/URLs?
 * Mendeley/Zotero/EndNote ID/URLs

Revelant Code and projects
Projects and gems that may be of interest
 * Anystyle Parser used primary for parsing citations, might be adapted to also parse microcitations
 * Citation Style Language Processor used to process various citation styles
 * OpenURL gem used to create, parse and use OpenURL queries eg to BHL or to CrossRef
 * GN Biblio tools, plug-ins some jQuery plugins to expose biblio parsing and mark-up
 * Pensoft/ViBRANT ReFinder API docs broken, code unknown
 * HathiTrust Research Center wiki tools to extract taxon names from literature
 * [http://www.bibtex.org/Convert/ BibTeX Converter
 * TeXMed - Exporting PubMed to BibTeX
 * [http://text2bib.economics.utoronto.ca/index.php/index text2bib

APIs available

 * The BHL API can be used to get bibliographic information on BHL pages and publications.
 * The Mendeley API can be used to get abstract, keywords, and search.
 * CrossRef Search API used to get all information about a DOI or search for citation, highly recommended
 * CrossRef OpenURL Send OpenURL requests to CrossRef (also API)
 * Open Library API and link to Ruby interface for the API

Links

 * Taeneonema example in BibTex
 * Taeneonema example as pdf bibliography