Lagotto 4.3 was released on July 20, 2015 with the following changes:
articlemetrics to lagotto (#397)The snapshots of the Lagotto JSON API require the same ENV variables as the CSV export. Run bundle exec rake api:snapshot:all to create the snapshots and export them to Zenodo. More information is available in the setup documentation.
This release adds foreign keys (supported in Rails core since Rails 4.2) to the database to enforce referential integrity.
Some sources now check for the work registration agency, allowing different functionality, e.g. for CrossRef or DataCite DOIs.
This release adds two sources that collect usage stats from the DataONE repository network. The dataone_counter source collects COUNTER-compliant usage stats, whereas the dataone_usage source also includes machine usage from some useragents.
Lagotto 4.2.1 was released on July 13, 2015 with the following changes:
Lagotto 4.2 was released on July 13, 2015 with the following changes:
The following new ENV variables must be set in the .env file to use the new Zenodo export functionality:
CREATOR=Public Library of Science
ZENODO_KEY=YOUR_KEY_HERE
ZENODO_URL=https://zenodo.org/api/
#ZENODO_URL=https://sandbox.zenodo.org/api/ for testing
SITENAMELONG="PLOS ALM"
A ZENODO_KEY can be obtained by creating a Zenodo account and then a personal access token on the profile page.
Lagotto 4.1.1 was released on July 6, 2015 with the following fixes:
Lagotto 4.1 was released on June 17, 2015 with the following changes:
/heartbeat endpoint to use sinatra so that it works if the database is down (#302)ENV variables (#326).rss (#328)europe_pmc_fulltext, plos_fulltext and bmc_fulltext (#336)This release includes numerous other small bug fixes. While the release fixes on bug fixes, new features were also introduced:
A list of similar works that we also track, generated by common relations, e.g. other works tracked by Lagotto cited
by the same paper or bookmarked in the same ORCID profile. Available in the v6 API, e.g. here.
Starting with this release, the id for a work in the v6 API will always be a URL, e.g. http://doi.org/10.1371/journal.pmed.1000097, and the use of dx.doi.org has been
changed to doi.org in all places.
Lagotto 4.1 adds support for second-order events, i.e. it will track news items and blog posts that discuss a work in Lagotto.
We do the same for scholarly articles that mention a tracked work in the body text (where we can search the fulltext), e.g. informal citations to datasets.
Lagotto 4.0 was released on May 5, 2015 with the following changes:
ads and ads_fulltext data sources. ads finds ArXiV preprints that became peer-reviewed papers, ads_fulltext allows fulltext search for all ArXiV preprints (#297)Starting with Lagotto 4.0 frontend assets are installed via NPM and Bower. This is done automatically when using Chef/Vagrant. When not using Chef/Vagarant please copy frontend/node_modules, frontend/bower_components, frontend/package.json, frontend/bower.json and frontend/.bowerrc into shared and run npm install (not npm -g install) from shared/frontend:
cd /var/www/lagotto/shared/frontend
mkdir node_modules
mkdir bower_components
wget https://raw.githubusercontent.com/lagotto/lagotto/4-0-stable/frontend/package.json
wget https://raw.githubusercontent.com/lagotto/lagotto/4-0-stable/frontend/bower.json
wget https://raw.githubusercontent.com/lagotto/lagotto/4-0-stable/frontend/.bowerrc
npm install
CouchDB is no longer needed for core Lagotto functionality, but is still used to store responses from some of the sources (e.g. F1000).
Make sure to run the data migrations in addition to the database migrations. This happens automatically if using capistrano, otherwise run rake data:migrate.
After upgrading the code and running the database migrations, old monthly and daily data stored in CouchDB should be imported via:
bundle exec rake couchdb:import
Starting with the new v6 API, API versions (currently defaulting to version 6) and authentication via API key (needed for admin functionality) are provided via the HTTP header, e.g. in curl:
- H "Accept: application/json; version=6" -H "Authorization: Token token=API_KEY"
Lagotto 3.20 was released on May 5, 2015 with the following changes:
twitter_search source (#300)Make sure to run the data migrations in addition to the database migrations. This happens automatically if using capistrano, otherwise run rake data:migrate.
Lagotto 3.19 was released on March 24, 2015 with the following changes:
published_id when creating works via the v4 API (#264)jasmine_rails to teaspoon for running javascript jasmine unit tests (#276)To customize the signposts, add/uncomment the following ENV variables in your .env file, separating multiple sources via comma:
# customize signposts, defaults are shown
# VIEWED=pmc,counter
# SAVED=mendeley,citeulike
# DISCUSSED=twitter,twitter_search,facebook
# CITED=crossref
Lagotto 3.18 was released on March 2, 2015 with the following changes:
pmc_europe, europe_pmc_fulltext, and wikipedia sources (#255)DB_PORT ENV variable, defaults to 3306 (#258)The live API documentation is available at /api.
To load the new nature_opensearch source please run
RAILS_ENV=production bundle exec rake db:seed
Lagotto 3.17 was released on February 19, 2015 with the following changes:
dataset parameter from Europe PMC fulltext search (#245)To load the new sources please run
RAILS_ENV=production bundle exec rake db:seed
Lagotto 3.16 was released on February 8, 2015 with the following changes:
The style guide is available at /docs/styleguide.
Lagotto 3.15 was released on February 4, 2015 with the following changes:
vagrant push command, using vagrant-capistrano-push plugin (#238)Make sure the following settings for the Mendeley source are corect:
Lagotto 3.14 was released on January 19, 2015 with the following change:
Lagotto 3.13 was released on January 15, 2015 with the following changes:
README.md (#224)shared folder instead of current for better compatibility with Capistrano (#227)The big change in this release is the switch from delayed_job to Sidekiq for background processing. Sidekiq requires the redis database that can be installed either via the updated Chef scripts, or manually, e.g. on Ubuntu:
sudo apt-get install redis-server
Also, make sure the following ENV variable is set in your .env file:
# number of threads Sidekiq uses
CONCURRENCY=25
No redis or sidekiq configuration is necessary, but make sure all delayed_job worker processes are killed when upgrading.
Lagotto 3.12.7 was released on January 9, 2015 with the following change:
Lagotto 3.12.6 was released on January 6, 2015 with the following change:
/heartbeat API endpoint, and changed status page accordingly (#226)Lagotto 3.12.5 was released on January 6, 2015 with the following change:
<i>, <b>, <sup>, <sub>, and <sc> HTML tags in work titles, following the CSL specification on rich text markup (#225)Lagotto 3.12.4 was released on January 5, 2015 with the following change:
Lagotto 3.12.3 was released on January 4, 2015 with the following change:
To load the new source please run
RAILS_ENV=production bundle exec rake db:seed
Lagotto 3.12.2 was released on January 4, 2015 with the following changes:
nil, using the nilify_blanks gem (#218)Lagotto 3.12.1 was released on January 4, 2015 with the following changes:
cron:nightlyrake task, and write to separate cron_import.log (#216)Starting with this release, the raw responses from external sources are stored in a JSON file log/agent.log in logstash format, using the source name and work pid as tags:
{
"message": "{\"total_rows\"=>328331, \"offset\"=>121903, \"rows\"=>[{\"id\"=>\"232944431433666561\", \"key\"=>\"10.1371/journal.pone.0042231\", \"value\"=>{\"_id\"=>\"232944431433666561\", \"_rev\"=>\"1-40047ae514c178f154fa7b0f877f146a\", \"text\"=>\"Role of the Irr Protein in the Regulation of Iron Metabolism in Rhodobacter sphaeroides http://t.co/JbOupbjK\", \"from_user_id\"=>38951828, \"from_user_name\"=>\"Test\", \"geo\"=>nil, \"profile_image_url_https\"=>\"https://si0.twimg.com/sticky/default_profile_images/default_profile_6_normal.png\", \"iso_language_code\"=>\"en\", \"to_user_name\"=>nil, \"entities\"=>{\"urls\"=>[{\"expanded_url\"=>\"http://bit.ly/O0DVOi\", \"indices\"=>[88, 108], \"display_url\"=>\"bit.ly/O0DVOi\", \"url\"=>\"http://t.co/JbOupbjK\"}, {\"expanded_url\"=>\"http://www.plosone.org/article/info:doi%2F10.1371%2Fjournal.pone.0042231\", \"display_url\"=>\"http://www.plosone.org/article/info:doi%2F10.1371%2Fjournal.pone.0042231\", \"url\"=>\"http://www.plosone.org/article/info:doi%2F10.1371%2Fjournal.pone.0042231\"}], \"hashtags\"=>[], \"user_mentions\"=>[]}, \"to_user_id\"=>0, \"id\"=>232944431433666560, \"to_user_id_str\"=>\"0\", \"source\"=>\"<a href="http://twitterfeed.com" rel="nofollow">twitterfeed</a>\", \"from_user_id_str\"=>\"38951828\", \"from_user\"=>\"TestCellBio\", \"created_at\"=>\"Tue, 07 Aug 2012 21:00:55 +0000\", \"to_user\"=>nil, \"id_str\"=>\"232944431433666561\", \"profile_image_url\"=>\"http://a0.twimg.com/sticky/default_profile_images/default_profile_6_normal.png\", \"metadata\"=>{\"result_type\"=>\"recent\"}}}]}",
"@timestamp": "2015-01-04T01:13:53.387-08:00",
"@version": "1",
"severity": "INFO",
"host": "rwc-prod-alm03",
"tags": [
"ActiveJob",
"SourceJob",
"2822afb7-1a37-4084-b3c2-b0b87f0370a0",
"twitter",
"10.1371/journal.pone.0042231"
]
}
This file can be further processed with logstash and made available for download.
Lagotto 3.12 was released on January 1, 2015 with the following changes:
retrieval_statuses.events_url database column to handle URLs longer than 255 characters (#205)db:articles:load task for backwards compatibility (#207)The automatic import of works - configured in the .env file - has changed:
# automatic import of works published on current or previous day
# using CrossRef, DataCite, or PLOS Search API, or no automatic import if left empty
# Possible parameters:
# crossref - all works in CrossRef REST API
# member - all works in CrossRef REST API for publishers registered in application
# sample - sample of 20 works from CrossRef REST API
# member_sample - sample of 20 works from CrossRef REST API for publishers registered in application
# datacite - all works in DataCite metadata index
# plos - all PLOS articles
IMPORT=
To load the new sources please run
RAILS_ENV=production bundle exec rake db:seed
Lagotto 3.11 was released on December 24, 2014 with the following changes:
By default Lagotto uses Mozilla Persona for authentication. To use any of the other authentication options (cas, orcid, github), provide the options in the .env file:
# authentication via orcid, github, cas or persona. Defaults to persona
OMNIAUTH=
GITHUB_CLIENT_ID=
GITHUB_CLIENT_SECRET=
ORCID_CLIENT_ID=
ORCID_CLIENT_SECRET=
CAS_URL=
CAS_INFO_URL=
CAS_PREFIX=
To load the new Github source please run
RAILS_ENV=production bundle exec rake db:seed
Lagotto 3.10 was released on December 18, 2014 with the following changes:
articles to works to make it clear that the software can track all scholarly outputs (#190)If you are upgrading and have used DOIs as persistent identifier, please run the following rake task to fill in the new pid_type and pid fields:
RAILS_ENV=production bundle exec rake db:works:load_pids
To load the renamed filters, please run
RAILS_ENV=production bundle exec rake db:seed
Lagotto 3.9.8 was released on December 18, 2014 with the following change:
Lagotto 3.9.7 was released on December 11, 2014 with the following change:
twitter_search, and make the search string configurable in the source settings (#189)Please change url in the twitter_search source to
https://api.twitter.com/1.1/search/tweets.json?q="%{doi}" OR "%{query_url}"&count=100&include_entities=1&result_type=recent
Change events_url to
https://twitter.com/search?q="%{doi}" OR "%{query_url}"&f=realtime
Lagotto 3.9.6 was released on December 5, 2014 with the following changes:
Lagotto 3.9.5 was released on December 4, 2014 with the following change:
ENV["LOGSTASH_TYPE"]=redis and ENV["LOGSTASH_HOST"]=example.com.Lagotto 3.9.4 was released on December 2, 2014 with the following change:
Delayed::Worker.sleep_delay (the delayed_jobs polling interval of MySQL) via ENV["DJ_SLEEP_DELAY"], defaulting to 5 sec (#188)Lagotto 3.9.3 was released on November 26, 2014 with the following change:
rack::deflater middleware and better use of Last-Modified header in v5 API (#187)Lagotto 3.9.2 was released on November 26, 2014 with the following change:
Lagotto 3.9.1 was released on November 8, 2014 with the following changes:
rake, rails, cap and whenever (#182)Lagotto 3.9 was released on November 7, 2014 with the following changes:
SECRET_KEY_BASE into the Rails 4.1 secrets.yml fileLagotto 3.8.1 was released on November 5, 2014 with the following changes:
...public/assets/manifest*': No such file or directory
LOG_LEVEL option for rails and capistrano to .env
Lagotto 3.8 was released on November 4, 2014 with the following changes:
Please change SECRET_TOKEN in your .env file to SECRET_KEY_BASE.
Lagotto 3.7.1 was released on October 31, 2014 with the following features and bug fixes:
.env files via DOTENV ENV variable, default to DOTENV=default (#174)Lagotto 3.7 was released on October 28, 2014 with the following features and bug fixes:
.env file. See below for more information (#146)"~> 3.6.3" (#155)app_key and app_secret. Since the release of the v2.1 API in August 2014 the link_stat API endpoint is depreciated. New user accounts have to use the v2.1 API and only get the total count of Facebook activity, whereas users will older API keys can still use the link_stat API and get the number of shares, comments and likes in addition to the total count by adding the following link_stat URL in the Facebook configuration:https://graph.facebook.com/fql?access_token=%{access_token}&q=select url, share_count, like_count, comment_count, click_count, total_count from link_stat where url = '%{query_url}'
Starting with the Lagotto 3.7 release all user-specific configuration options for Rails, as well as for the server configuration and deployment tools Vagrant, Chef and Capistrano are environment variables, and can be stored in a single .env file. An example file is provided (.env.example) and can be used without modifications for a development server. More information regarding ENV variables and .env is available here. The following configuration options need to be set:
# Example configuration settings for this application
APPLICATION=lagotto
# database settings
DB_NAME=lagotto
DB_USERNAME=vagrant
DB_PASSWORD=
DB_HOST=localhost
# internal name of server
HOSTNAME=lagotto.local
# public name of server
# can be HOSTNAME, or different if load balancer is used
SERVERNAME=lagotto.local
# all instances of server used behind load balancer
# can be HOSTNAME, or comma-delimited string of HOSTNAME
SERVERS=lagotto.local
# name used on navigation bar and in email subject line
SITENAME=ALM
# couch_db database
COUCHDB_URL=http://localhost:5984/lagotto
# email address for sending emails
ADMIN_EMAIL=admin@example.com
# number of background workers
WORKERS=3
# automatic import via CrossRef API.
# Use 'all', 'member', 'sample', 'member_sample', or leave empty
IMPORT=
# persistent identifier used
UID=doi
# keys
# run `rake secret` to generate these keys
API_KEY=8897f9349100728d66d64d56bc21254bb346a9ed21954933
SECRET_TOKEN=c436de247c988eb5d0908407e700098fc3992040629bb8f98223cd221e94ee4d15626aae5d815f153f3dbbce2724ccb8569c4e26a0f6f663375f6f2697f1f3cf
# mail settings
MAIL_ADDRESS=localhost
MAIL_PORT=25
MAIL_DOMAIN=localhost
# vagrant settings
PRIVATE_IP=10.2.2.4
AWS_KEY=
AWS_SECRET=
AWS_KEYNAME=
AWS_KEYPATH=
DO_PROVIDER_TOKEN=
DO_SIZE=1GB
SSH_PRIVATE_KEY='~/.ssh/id_rsa'
# user and group who own application repository
DEPLOY_USER=vagrant
DEPLOY_GROUP=vagrant
# mysql server root password for chef
DB_SERVER_ROOT_PASSWORD=EZ$zspyxF2
The Mendeley configuration variable secret has been renamed to client_secret to be more consistent with other OAuth2 applications, please update your configuration.
Lagotto 3.6.3 was released on October 12, 2014 with the following features:
Lagotto 3.6.2 was released on October 10, 2014 with the following features:
Lagotto 3.6 was released on October 8, 2014 with the following features and bugfixes:
Users upgrading from earlier versions need to make the following changes:
CrossRef and PMC source. Add Openurl username to CrossRef if you plan to collect citation counts for articles where you are not the publisher.With this release the ALM application was renamed to Lagotto and the license changed
from Apache 2.0 to a MIT license. Lagotto 3.5 was released on September 14, 2014
with the following features and bugfixes:
ALM 3.4.8 was released on August 28, 2014 with the following bugfix:
ALM 3.4.2 was released on August 24, 2014 with the following bugfix:
ALM 3.4 was released on August 22, 2014 with the following new features:
INFO, WARN, ERRORand FATAL
/heartbeat API endpoint to check application healthTo enable the automated daily article import via CrossRef API, add the following line to config/settings.yml:
import: member
To import articles manually, leave import empty; or use one of the following options for automated daily import via cron:
import: all: All DOIs updated the last dayimport: sample: A random sample of 20 from all DOIs updated the last dayimport: member: All DOIs from the publishers in the publisher admin interface, updated the last dayimport: member_sample: A random sample of 20 from all DOIs from the publishers in the publisher admin interface, updated the last dayPublishers can now be added in the admin interface. The only functionality is currently the automated article import described above.
For admin users alerts are now available via API, use the /api/v4/alerts endpoint and basic authentication. More details in the API documentation.
There is one configuration change in error reporting: the disabled_source_report has been renamed to fatal_error_report, as an immediate email will now be sent for all errors with severity FATAL. To rename the report template in the database, run this rake task once (using the appropriate RAILS_ENV):
bundle exec rake mailer:rename_report RAILS_ENV=production
ALM 3.3.19 was released on August 18, 2014 with the following new feature:
ALM 3.3.14 was released on August 15, 2014 with the following bugfix:
ALM 3.3.12 was released on August 13, 2014 with the following new feature:
START_DATE and END_DATE.bundle exec rake couchdb:histories:delete START_DATE=2014-01-01
ALM 3.3.8 was released on August 8, 2014 with the following bugfix:
ALM 3.3.5 was released on August 5, 2014 with the following bugfix:
ALM 3.3.2 was released on August 3, 2014 with the following bugfixes:
ALM 3.3.1 was released on July 31, 2014 with the following bugfix:
ALM 3.3 was released on July 29, 2014 with the following features:
ALM 3.2 was released on July 1, 2014 with the following features:
ALM 3.1 was released on May 23, 2014 with the following features:
ALM 3.0 was released on May 8, 2014 with the following features:
ALM 2.14 was released on April 24, 2014 with the following new features:
ALM 2.13.2 was released on March 27, 2014 with a focus on bug fixes:
Migrate to new date format (year, month, day):
bundle exec rake db:articles:date_parts RAILS_ENV=production
Add number of workers in config/settings.yml:
defaults: &defaults
workers: 3
Update Mendeley configuration (after getting client ID and secret from Mendeley):
Url
https://api-oauth2.mendeley.com/oapi/documents/details/%{id}
Url with type
https://api-oauth2.mendeley.com/oapi/documents/details/%{id}/?type=%{doc_type}
Url with title
https://api-oauth2.mendeley.com/oapi/documents/search/%{title}/?items=10
Authentication url
https://api-oauth2.mendeley.com/oauth/token
Related articles url
https://api-oauth2.mendeley.com/oapi/documents/related/%{id}
Client id
EXAMPLE
Secret
EXAMPLE
Add Scopus source (after getting api_key and insttoken from Scopus):
api_key
EXAMPLE
insttoken
EXAMPLE
ALM 2.12.1 was released on February 10, 2014 with the following changes:
Update Facebook configuration:
URL
https://graph.facebook.com/fql?access_token=%{access_token}&q=select url, share_count, like_count, comment_count, click_count, total_count from link_stat where url = '%{query_url}'
ALM 2.11.2 was released on January 22, 2014. Changes in this release include:
Update CouchDB design docs (needed for reports):
curl -X PUT -d @design-doc/reports.json 'http://localhost:5984/alm/_design/reports'
Remove HTML and XML from article titles:
bundle exec db:articles:sanitize_title
ALM 2.10.1 was released on November 13, 2013 with the following features:
ALM 2.9.15 was released on November 3, 2013. This release contains fixes for the mailer functionality and two new reports:
A new source has been added:
ALM 2.9.7 was released on October 3, 2013. This release fixes many small bugs of the 2.9 release and adds the following new sources:
ALM 2.9 was released on September 16, 2013. This version contains numerous bug fixes, and some important parts of the application were refactored
The major new feature in this release is error tracking via filters. Filters can easily be added and customized, and they will generate a daily error report that is sent out via email
ALM 2.8 was released on July 22, 2013. The development work focused on user account management.
Prior to this release, the ALM application allowed only a single user account. We can now have multiple user accounts, plus different roles for them:
A new user admin dashboard facilitates user management.
To facilitate account management for users and ALM admins, new user accounts can only be created via social login with one of these services:
The ALM application can be configured to use one of these services (but not both), or to use the old system of only a single admin user. The documentation has been updated to reflect these changes.
Now that API keys are created within the ALM application, we can check for valid API keys in every API request. This is currently only done to allow access to private sources for admins, and to enable create/update/delete in the REST API. For security reasons create/update/delete only works from requests originating from the same computer.
The documentation in the wiki is now included in the application, making it easier for users to find documentation. We use the github-markdown gem for this. The documentation has been updated with the changes in ALM 2.8, and other updates.
Many small changes and bug fixes. We now use memcached for caching, and we added to cron jobs to clean up temporary tables (error_messages and api_requests).
ALM 2.7 was released on May 16, 2013. The development work focused on adding Javascript libraries that talk to the ALM API and can be embedded in other web sites.
We added visualizations for single articles using the d3.js library, including time-based visualizations by day, month and year. We have created a new Github repository that will hold all future work on Javascript libraries talking to the ALM API. To make these visualizations easier we made two changes to the v3 ALM API:
We have made it easier to install the ALM application by adding and testing support for Amazon AWS via Vagrant. We added rake tasks to seed or delete articles, and to delete resolved errors and old API requests. Many small bugs were fixed and the ALM application was updated to use the latest Rails version (3.2.13).
ALM 2.6 was released on March 19, 2013. The development work focused on two areas: API performance and easy installation.
The API code was refactored, adding the Draper Decorator between the model and the RABL Views. We added caching, using Russian Doll fragment caching as described here. It required some extra work to make this work with RABL and Draper. We also added a visualization of API requests to the admin dashboard, using d3.js and the crossfiler library.
To make it easier to install the ALM application, we have updated and throughoutly tested the manual installation instructions, updated the Chef / Vagrant automated installation (now using Ubuntu 12.04 and linking directly to the Chef Cookbook repository), and provided a VMware image of ALM application (535 MB download, username/password: alm). The VMware image file is still experimental and feedback is appreciated.
An updated version of Vagrant (1.1) was released days after ALM 2.6, and this version now supports VMware and AWS. We will support these platforms in a future ALM release to again make it easier to test or install the ALM application.
This release includes many small fixes and improvements. Of particular interest are the RSS feeds for the most popular articles by source, published in the last 7 days, 30 days, 12 months, or all-time. The RSS feeds are link from the most-cited lists for each source, e.g. here.
ALM 2.5 was released on February 1, 2013.
ALM 2.4 was released on December 20, 2012.
In this release we added a new source (ScienceSeeker, a blog aggregator), fix many errors with sources and added tests for background and Rake tasks.
ALM 2.3 was released on October 30, 2012.
ALM 2.2 was released on October 5, 2012.
ALM 2.1 was released on September 13, 2012.
ALM 2.0 was released on July 31, 2012.