Reinforcement Learning Using a Continuous Time Actor-Critic Framework with Spiking Neurons
Events
Loading … Spinner

CiteULike | Further Information

Mendeley | Further Information

{"title"=>"Reinforcement Learning Using a Continuous Time Actor-Critic Framework with Spiking Neurons", "type"=>"journal", "authors"=>[{"first_name"=>"Nicolas", "last_name"=>"Frémaux", "scopus_author_id"=>"35317422500"}, {"first_name"=>"Henning", "last_name"=>"Sprekeler", "scopus_author_id"=>"20735866700"}, {"first_name"=>"Wulfram", "last_name"=>"Gerstner", "scopus_author_id"=>"56264524000"}], "year"=>2013, "source"=>"PLoS Computational Biology", "identifiers"=>{"issn"=>"1553734X", "scopus"=>"2-s2.0-84876888983", "pmid"=>"23592970", "doi"=>"10.1371/journal.pcbi.1003024", "pui"=>"368832130", "isbn"=>"1553-734X", "sgr"=>"84876888983"}, "id"=>"2e280249-33aa-3cf9-8ab3-9b2329a2297c", "abstract"=>"Animals repeat rewarded behaviors, but the physiological basis of reward-based learning has only been partially elucidated. On one hand, experimental evidence shows that the neuromodulator dopamine carries information about rewards and affects synaptic plasticity. On the other hand, the theory of reinforcement learning provides a framework for reward-based learning. Recent models of reward-modulated spike-timing-dependent plasticity have made first steps towards bridging the gap between the two approaches, but faced two problems. First, reinforcement learning is typically formulated in a discrete framework, ill-adapted to the description of natural situations. Second, biologically plausible models of reward-modulated spike-timing-dependent plasticity require precise calculation of the reward prediction error, yet it remains to be shown how this can be computed by neurons. Here we propose a solution to these problems by extending the continuous temporal difference (TD) learning of Doya (2000) to the case of spiking neurons in an actor-critic network operating in continuous time, and with continuous state and action representations. In our model, the critic learns to predict expected future rewards in real time. Its activity, together with actual rewards, conditions the delivery of a neuromodulatory TD signal to itself and to the actor, which is responsible for action choice. In simulations, we show that such an architecture can solve a Morris water-maze-like navigation task, in a number of trials consistent with reported animal performance. We also use our model to solve the acrobot and the cartpole problems, two complex motor control tasks. Our model provides a plausible way of computing reward prediction error in the brain. Moreover, the analytically derived learning rule is consistent with experimental evidence for dopamine-modulated spike-timing-dependent plasticity.", "link"=>"http://www.mendeley.com/research/reinforcement-learning-using-continuous-time-actorcritic-framework-spiking-neurons-2", "reader_count"=>179, "reader_count_by_academic_status"=>{"Unspecified"=>2, "Professor > Associate Professor"=>5, "Researcher"=>45, "Student > Doctoral Student"=>4, "Student > Ph. D. Student"=>63, "Student > Postgraduate"=>7, "Student > Master"=>34, "Other"=>1, "Student > Bachelor"=>10, "Lecturer"=>2, "Professor"=>6}, "reader_count_by_user_role"=>{"Unspecified"=>2, "Professor > Associate Professor"=>5, "Researcher"=>45, "Student > Doctoral Student"=>4, "Student > Ph. D. Student"=>63, "Student > Postgraduate"=>7, "Student > Master"=>34, "Other"=>1, "Student > Bachelor"=>10, "Lecturer"=>2, "Professor"=>6}, "reader_count_by_subject_area"=>{"Engineering"=>28, "Unspecified"=>3, "Mathematics"=>7, "Agricultural and Biological Sciences"=>43, "Medicine and Dentistry"=>6, "Neuroscience"=>19, "Arts and Humanities"=>1, "Philosophy"=>1, "Physics and Astronomy"=>7, "Psychology"=>7, "Chemistry"=>1, "Computer Science"=>56}, "reader_count_by_subdiscipline"=>{"Engineering"=>{"Engineering"=>28}, "Medicine and Dentistry"=>{"Medicine and Dentistry"=>6}, "Neuroscience"=>{"Neuroscience"=>19}, "Chemistry"=>{"Chemistry"=>1}, "Physics and Astronomy"=>{"Physics and Astronomy"=>7}, "Psychology"=>{"Psychology"=>7}, "Agricultural and Biological Sciences"=>{"Agricultural and Biological Sciences"=>43}, "Computer Science"=>{"Computer Science"=>56}, "Mathematics"=>{"Mathematics"=>7}, "Unspecified"=>{"Unspecified"=>3}, "Arts and Humanities"=>{"Arts and Humanities"=>1}, "Philosophy"=>{"Philosophy"=>1}}, "reader_count_by_country"=>{"United States"=>5, "Japan"=>1, "United Kingdom"=>7, "Switzerland"=>5, "Greece"=>1, "New Zealand"=>1, "Canada"=>1, "Netherlands"=>1, "Sweden"=>1, "Austria"=>1, "Turkey"=>1, "China"=>1, "Poland"=>1, "Italy"=>1, "France"=>3, "Germany"=>7}, "group_count"=>8}

CrossRef

Scopus | Further Information

{"@_fa"=>"true", "link"=>[{"@_fa"=>"true", "@ref"=>"self", "@href"=>"https://api.elsevier.com/content/abstract/scopus_id/84876888983"}, {"@_fa"=>"true", "@ref"=>"author-affiliation", "@href"=>"https://api.elsevier.com/content/abstract/scopus_id/84876888983?field=author,affiliation"}, {"@_fa"=>"true", "@ref"=>"scopus", "@href"=>"https://www.scopus.com/inward/record.uri?partnerID=HzOxMe3b&scp=84876888983&origin=inward"}, {"@_fa"=>"true", "@ref"=>"scopus-citedby", "@href"=>"https://www.scopus.com/inward/citedby.uri?partnerID=HzOxMe3b&scp=84876888983&origin=inward"}], "prism:url"=>"https://api.elsevier.com/content/abstract/scopus_id/84876888983", "dc:identifier"=>"SCOPUS_ID:84876888983", "eid"=>"2-s2.0-84876888983", "dc:title"=>"Reinforcement Learning Using a Continuous Time Actor-Critic Framework with Spiking Neurons", "dc:creator"=>"Frémaux N.", "prism:publicationName"=>"PLoS Computational Biology", "prism:issn"=>"1553734X", "prism:eIssn"=>"15537358", "prism:volume"=>"9", "prism:issueIdentifier"=>"4", "prism:pageRange"=>nil, "prism:coverDate"=>"2013-04-01", "prism:coverDisplayDate"=>"April 2013", "prism:doi"=>"10.1371/journal.pcbi.1003024", "citedby-count"=>"55", "affiliation"=>[{"@_fa"=>"true", "affilname"=>"Ecole Polytechnique Fédérale de Lausanne", "affiliation-city"=>"Lausanne", "affiliation-country"=>"Switzerland"}], "pubmed-id"=>"23592970", "prism:aggregationType"=>"Journal", "subtype"=>"ar", "subtypeDescription"=>"Article", "article-number"=>"e1003024", "source-id"=>"4000151810", "openaccess"=>"1", "openaccessFlag"=>true}

Facebook

  • {"url"=>"http%3A%2F%2Fjournals.plos.org%2Fploscompbiol%2Farticle%3Fid%3D10.1371%252Fjournal.pcbi.1003024", "share_count"=>0, "like_count"=>0, "comment_count"=>0, "click_count"=>0, "total_count"=>0}

Twitter

Counter

  • {"month"=>"4", "year"=>"2013", "pdf_views"=>"222", "xml_views"=>"65", "html_views"=>"1286"}
  • {"month"=>"5", "year"=>"2013", "pdf_views"=>"73", "xml_views"=>"13", "html_views"=>"510"}
  • {"month"=>"6", "year"=>"2013", "pdf_views"=>"40", "xml_views"=>"0", "html_views"=>"164"}
  • {"month"=>"7", "year"=>"2013", "pdf_views"=>"25", "xml_views"=>"1", "html_views"=>"141"}
  • {"month"=>"8", "year"=>"2013", "pdf_views"=>"36", "xml_views"=>"2", "html_views"=>"108"}
  • {"month"=>"9", "year"=>"2013", "pdf_views"=>"22", "xml_views"=>"3", "html_views"=>"98"}
  • {"month"=>"10", "year"=>"2013", "pdf_views"=>"23", "xml_views"=>"2", "html_views"=>"159"}
  • {"month"=>"11", "year"=>"2013", "pdf_views"=>"11", "xml_views"=>"0", "html_views"=>"74"}
  • {"month"=>"12", "year"=>"2013", "pdf_views"=>"20", "xml_views"=>"1", "html_views"=>"94"}
  • {"month"=>"1", "year"=>"2014", "pdf_views"=>"8", "xml_views"=>"1", "html_views"=>"102"}
  • {"month"=>"2", "year"=>"2014", "pdf_views"=>"28", "xml_views"=>"1", "html_views"=>"92"}
  • {"month"=>"3", "year"=>"2014", "pdf_views"=>"13", "xml_views"=>"1", "html_views"=>"80"}
  • {"month"=>"4", "year"=>"2014", "pdf_views"=>"22", "xml_views"=>"1", "html_views"=>"101"}
  • {"month"=>"5", "year"=>"2014", "pdf_views"=>"19", "xml_views"=>"0", "html_views"=>"51"}
  • {"month"=>"6", "year"=>"2014", "pdf_views"=>"14", "xml_views"=>"0", "html_views"=>"56"}
  • {"month"=>"7", "year"=>"2014", "pdf_views"=>"18", "xml_views"=>"1", "html_views"=>"72"}
  • {"month"=>"8", "year"=>"2014", "pdf_views"=>"19", "xml_views"=>"3", "html_views"=>"59"}
  • {"month"=>"9", "year"=>"2014", "pdf_views"=>"21", "xml_views"=>"0", "html_views"=>"68"}
  • {"month"=>"10", "year"=>"2014", "pdf_views"=>"30", "xml_views"=>"1", "html_views"=>"87"}
  • {"month"=>"11", "year"=>"2014", "pdf_views"=>"11", "xml_views"=>"2", "html_views"=>"73"}
  • {"month"=>"12", "year"=>"2014", "pdf_views"=>"16", "xml_views"=>"2", "html_views"=>"83"}
  • {"month"=>"1", "year"=>"2015", "pdf_views"=>"16", "xml_views"=>"0", "html_views"=>"91"}
  • {"month"=>"2", "year"=>"2015", "pdf_views"=>"21", "xml_views"=>"1", "html_views"=>"84"}
  • {"month"=>"3", "year"=>"2015", "pdf_views"=>"44", "xml_views"=>"0", "html_views"=>"110"}
  • {"month"=>"4", "year"=>"2015", "pdf_views"=>"18", "xml_views"=>"1", "html_views"=>"63"}
  • {"month"=>"5", "year"=>"2015", "pdf_views"=>"20", "xml_views"=>"0", "html_views"=>"98"}
  • {"month"=>"6", "year"=>"2015", "pdf_views"=>"12", "xml_views"=>"0", "html_views"=>"46"}
  • {"month"=>"7", "year"=>"2015", "pdf_views"=>"26", "xml_views"=>"0", "html_views"=>"44"}
  • {"month"=>"8", "year"=>"2015", "pdf_views"=>"6", "xml_views"=>"0", "html_views"=>"66"}
  • {"month"=>"9", "year"=>"2015", "pdf_views"=>"12", "xml_views"=>"1", "html_views"=>"50"}
  • {"month"=>"10", "year"=>"2015", "pdf_views"=>"8", "xml_views"=>"0", "html_views"=>"65"}
  • {"month"=>"11", "year"=>"2015", "pdf_views"=>"13", "xml_views"=>"0", "html_views"=>"79"}
  • {"month"=>"12", "year"=>"2015", "pdf_views"=>"9", "xml_views"=>"0", "html_views"=>"43"}
  • {"month"=>"1", "year"=>"2016", "pdf_views"=>"9", "xml_views"=>"0", "html_views"=>"64"}
  • {"month"=>"2", "year"=>"2016", "pdf_views"=>"20", "xml_views"=>"0", "html_views"=>"60"}
  • {"month"=>"3", "year"=>"2016", "pdf_views"=>"13", "xml_views"=>"0", "html_views"=>"74"}
  • {"month"=>"4", "year"=>"2016", "pdf_views"=>"17", "xml_views"=>"0", "html_views"=>"95"}
  • {"month"=>"5", "year"=>"2016", "pdf_views"=>"20", "xml_views"=>"0", "html_views"=>"95"}
  • {"month"=>"6", "year"=>"2016", "pdf_views"=>"25", "xml_views"=>"0", "html_views"=>"138"}
  • {"month"=>"7", "year"=>"2016", "pdf_views"=>"33", "xml_views"=>"0", "html_views"=>"42"}
  • {"month"=>"8", "year"=>"2016", "pdf_views"=>"45", "xml_views"=>"0", "html_views"=>"43"}
  • {"month"=>"9", "year"=>"2016", "pdf_views"=>"48", "xml_views"=>"0", "html_views"=>"50"}
  • {"month"=>"10", "year"=>"2016", "pdf_views"=>"42", "xml_views"=>"0", "html_views"=>"52"}
  • {"month"=>"11", "year"=>"2016", "pdf_views"=>"38", "xml_views"=>"0", "html_views"=>"56"}
  • {"month"=>"12", "year"=>"2016", "pdf_views"=>"42", "xml_views"=>"0", "html_views"=>"162"}
  • {"month"=>"1", "year"=>"2017", "pdf_views"=>"33", "xml_views"=>"0", "html_views"=>"53"}
  • {"month"=>"2", "year"=>"2017", "pdf_views"=>"46", "xml_views"=>"0", "html_views"=>"123"}
  • {"month"=>"3", "year"=>"2017", "pdf_views"=>"57", "xml_views"=>"0", "html_views"=>"123"}
  • {"month"=>"4", "year"=>"2017", "pdf_views"=>"20", "xml_views"=>"0", "html_views"=>"107"}
  • {"month"=>"5", "year"=>"2017", "pdf_views"=>"31", "xml_views"=>"1", "html_views"=>"82"}
  • {"month"=>"6", "year"=>"2017", "pdf_views"=>"29", "xml_views"=>"0", "html_views"=>"64"}
  • {"month"=>"7", "year"=>"2017", "pdf_views"=>"24", "xml_views"=>"0", "html_views"=>"104"}
  • {"month"=>"8", "year"=>"2017", "pdf_views"=>"26", "xml_views"=>"0", "html_views"=>"122"}
  • {"month"=>"9", "year"=>"2017", "pdf_views"=>"21", "xml_views"=>"1", "html_views"=>"127"}
  • {"month"=>"10", "year"=>"2017", "pdf_views"=>"33", "xml_views"=>"3", "html_views"=>"105"}
  • {"month"=>"11", "year"=>"2017", "pdf_views"=>"61", "xml_views"=>"1", "html_views"=>"174"}
  • {"month"=>"12", "year"=>"2017", "pdf_views"=>"17", "xml_views"=>"0", "html_views"=>"98"}
  • {"month"=>"1", "year"=>"2018", "pdf_views"=>"38", "xml_views"=>"0", "html_views"=>"82"}
  • {"month"=>"2", "year"=>"2018", "pdf_views"=>"16", "xml_views"=>"0", "html_views"=>"48"}
  • {"month"=>"3", "year"=>"2018", "pdf_views"=>"48", "xml_views"=>"0", "html_views"=>"75"}
  • {"month"=>"4", "year"=>"2018", "pdf_views"=>"113", "xml_views"=>"0", "html_views"=>"47"}
  • {"month"=>"5", "year"=>"2018", "pdf_views"=>"30", "xml_views"=>"0", "html_views"=>"36"}
  • {"month"=>"6", "year"=>"2018", "pdf_views"=>"43", "xml_views"=>"0", "html_views"=>"60"}
  • {"month"=>"7", "year"=>"2018", "pdf_views"=>"30", "xml_views"=>"3", "html_views"=>"42"}
  • {"month"=>"8", "year"=>"2018", "pdf_views"=>"29", "xml_views"=>"1", "html_views"=>"29"}
  • {"month"=>"9", "year"=>"2018", "pdf_views"=>"39", "xml_views"=>"0", "html_views"=>"40"}
  • {"month"=>"10", "year"=>"2018", "pdf_views"=>"31", "xml_views"=>"1", "html_views"=>"42"}
  • {"month"=>"11", "year"=>"2018", "pdf_views"=>"25", "xml_views"=>"0", "html_views"=>"50"}
  • {"month"=>"12", "year"=>"2018", "pdf_views"=>"37", "xml_views"=>"1", "html_views"=>"28"}
  • {"month"=>"1", "year"=>"2019", "pdf_views"=>"33", "xml_views"=>"0", "html_views"=>"59"}
  • {"month"=>"2", "year"=>"2019", "pdf_views"=>"33", "xml_views"=>"0", "html_views"=>"51"}
  • {"month"=>"3", "year"=>"2019", "pdf_views"=>"40", "xml_views"=>"4", "html_views"=>"53"}
  • {"month"=>"4", "year"=>"2019", "pdf_views"=>"36", "xml_views"=>"0", "html_views"=>"43"}
  • {"month"=>"5", "year"=>"2019", "pdf_views"=>"52", "xml_views"=>"0", "html_views"=>"41"}
  • {"month"=>"6", "year"=>"2019", "pdf_views"=>"35", "xml_views"=>"1", "html_views"=>"39"}
  • {"month"=>"7", "year"=>"2019", "pdf_views"=>"64", "xml_views"=>"0", "html_views"=>"80"}
  • {"month"=>"8", "year"=>"2019", "pdf_views"=>"43", "xml_views"=>"0", "html_views"=>"30"}
  • {"month"=>"9", "year"=>"2019", "pdf_views"=>"40", "xml_views"=>"0", "html_views"=>"43"}
  • {"month"=>"10", "year"=>"2019", "pdf_views"=>"58", "xml_views"=>"0", "html_views"=>"59"}
  • {"month"=>"11", "year"=>"2019", "pdf_views"=>"45", "xml_views"=>"0", "html_views"=>"50"}
  • {"month"=>"12", "year"=>"2019", "pdf_views"=>"26", "xml_views"=>"0", "html_views"=>"49"}
  • {"month"=>"1", "year"=>"2020", "pdf_views"=>"33", "xml_views"=>"1", "html_views"=>"30"}
  • {"month"=>"2", "year"=>"2020", "pdf_views"=>"18", "xml_views"=>"1", "html_views"=>"28"}
  • {"month"=>"3", "year"=>"2020", "pdf_views"=>"26", "xml_views"=>"2", "html_views"=>"48"}
  • {"month"=>"4", "year"=>"2020", "pdf_views"=>"64", "xml_views"=>"1", "html_views"=>"46"}
  • {"month"=>"5", "year"=>"2020", "pdf_views"=>"53", "xml_views"=>"0", "html_views"=>"41"}
  • {"month"=>"6", "year"=>"2020", "pdf_views"=>"30", "xml_views"=>"0", "html_views"=>"58"}
  • {"month"=>"7", "year"=>"2020", "pdf_views"=>"32", "xml_views"=>"1", "html_views"=>"52"}
  • {"month"=>"8", "year"=>"2020", "pdf_views"=>"54", "xml_views"=>"1", "html_views"=>"61"}
  • {"month"=>"9", "year"=>"2020", "pdf_views"=>"60", "xml_views"=>"0", "html_views"=>"61"}
  • {"month"=>"10", "year"=>"2020", "pdf_views"=>"63", "xml_views"=>"1", "html_views"=>"72"}
  • {"month"=>"11", "year"=>"2020", "pdf_views"=>"45", "xml_views"=>"0", "html_views"=>"51"}

Figshare

  • {"files"=>["https://ndownloader.figshare.com/files/1022035"], "description"=>"<p>From bottom to top: the simulated agent evolves in a maze environment, until it finds the reward area (green disk), avoiding obstacles (red). Place cells maintain a representation of the position of the agent through their tuning curves. Blue shadow: example tuning curve of one place cell (black); blue dots: tuning curves centers of other place cells. Right: a pool of critic neurons encode the expected future reward (value map, top right) at the agent's current position. The change in the predicted value is compared to the actual reward, leading to the temporal difference (TD) error. The TD error signal is broadcast to the synapses as part of the learning rule. Left: a ring of actor neurons with global inhibition and local excitation code for the direction taken by the agent. Their choices depending on the agent's position embody a policy map (top left).</p>", "links"=>[], "tags"=>["neuroscience", "computational neuroscience", "Learning and memory", "actor-critic"], "article_id"=>680536, "categories"=>["Biological Sciences"], "users"=>["Nicolas Frémaux", "Henning Sprekeler", "Wulfram Gerstner"], "doi"=>["https://dx.doi.org/10.1371/journal.pcbi.1003024.g001"], "stats"=>{"downloads"=>0, "page_views"=>0, "likes"=>0}, "figshare_url"=>"https://figshare.com/articles/_Navigation_task_and_actor_critic_network_/680536", "title"=>"Navigation task and actor-critic network.", "pos_in_sequence"=>0, "defined_type"=>1, "published_date"=>"2013-04-11 00:08:56"}
  • {"files"=>["https://ndownloader.figshare.com/files/1022036"], "description"=>"<p>A: Learning rule with three factors. Top: TD-LTP is the learning rule given in <a href=\"http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1003024#pcbi.1003024.e133\" target=\"_blank\">Eq. 17</a>. It works by passing the presynaptic spike train (factor 1) and the postsynaptic spike train (factor 2) through a coincidence window . Spikes are counted as coincident if the postsynaptic spike occurs within after a few ms of a presynaptic spike. The result of the pre-post coincidence measure is filtered through a kernel, and then multiplied by the TD error (factor 3) to yield the learning rule which controls the change of the synaptic weight. Bottom: TD-STDP is a TD-modulated variant of R-STDP. The main difference with TD-LTP is the presence of a post-before-pre component in the coincidence window. B: Linear track task. The linear track experiment is a simplified version of the standard maze task. The actor's choice is forced to the correct direction with constant velocity (left), while the critic learns to represent value (right). C: Value function learning by the critic. Each colored trace shows the value function represented by the critic neurons activity against time in the first simulation trials (from dark blue in trial 1 to dark red in trial 20), with corresponding to the time of the reward delivery. The black line shows an average over trials 30 to 50, after learning converged. The gray dashed line shows the theoretical value function. D: TD signal corresponding to the simulation in C. The gray dashed line shows the reward time course .</p>", "links"=>[], "tags"=>["neuroscience", "computational neuroscience", "Learning and memory", "linear"], "article_id"=>680537, "categories"=>["Biological Sciences"], "users"=>["Nicolas Frémaux", "Henning Sprekeler", "Wulfram Gerstner"], "doi"=>["https://dx.doi.org/10.1371/journal.pcbi.1003024.g002"], "stats"=>{"downloads"=>0, "page_views"=>0, "likes"=>0}, "figshare_url"=>"https://figshare.com/articles/_Critic_learning_in_a_linear_track_task_/680537", "title"=>"Critic learning in a linear track task.", "pos_in_sequence"=>0, "defined_type"=>1, "published_date"=>"2013-04-11 00:08:57"}
  • {"files"=>["https://ndownloader.figshare.com/files/1022037"], "description"=>"<p>A: A ring of actor neurons with lateral connectivity (bottom, green: excitatory, red: inhibitory) embodies the agent's policy (top). B: Lateral connectivity. Each neuron codes for a distinct motion direction. Neurons form excitatory synapses to similarly tuned neurons and inhibitory synapses to other neurons. C: Activity of actor neurons during an example trial. The activity of the neurons (vertical axis) is shown as a color map against time (horizontal axis). The lateral connectivity ensures that there is a single bump of activity at every moment in time. The black line shows the direction of motion (right axis; arrows in panel B) chosen as a result of the neural activity. D: Maze trajectory corresponding to the trial shown in C. The numbered position markers match the times marked in C.</p>", "links"=>[], "tags"=>["neuroscience", "computational neuroscience", "Learning and memory"], "article_id"=>680538, "categories"=>["Biological Sciences"], "users"=>["Nicolas Frémaux", "Henning Sprekeler", "Wulfram Gerstner"], "doi"=>["https://dx.doi.org/10.1371/journal.pcbi.1003024.g003"], "stats"=>{"downloads"=>0, "page_views"=>0, "likes"=>0}, "figshare_url"=>"https://figshare.com/articles/_Actor_neurons_/680538", "title"=>"Actor neurons.", "pos_in_sequence"=>0, "defined_type"=>1, "published_date"=>"2013-04-11 00:08:58"}
  • {"files"=>["https://ndownloader.figshare.com/files/1022038"], "description"=>"<p>A: The maze consists of a square enclosure, with a circular goal area (green) in the center. A U-shaped obstacle (red) makes the task harder by forcing turns on trajectories from three out of the four possible starting locations (crosses). B: Color-coded trajectories of an example TD-LTP agent during the first 75 simulated trials. Early trials (blue) are spent exploring the maze and the obstacles, while later trials (green to red) exploit stereotypical behavior. C: Value map (color map) and policy (vector field) represented by the synaptic weights of the agent of panel B after 2000s simulated seconds. D: Goal reaching latency of agents using different learning rules. Latencies of simulated agents per learning rule are binned by 5 trials (trials 1–5, trials 6–10, etc.). The solid lines shows the median of the latencies for each trial bin and the shaded area represents the 25th to 75th percentiles. For the R-max rule these all fall in the time limit after which a trial was interrupted if the goal was not reached. The R-max agent were simulated without a critic (see main text).</p>", "links"=>[], "tags"=>["neuroscience", "computational neuroscience", "Learning and memory", "navigation"], "article_id"=>680539, "categories"=>["Biological Sciences"], "users"=>["Nicolas Frémaux", "Henning Sprekeler", "Wulfram Gerstner"], "doi"=>["https://dx.doi.org/10.1371/journal.pcbi.1003024.g004"], "stats"=>{"downloads"=>0, "page_views"=>0, "likes"=>0}, "figshare_url"=>"https://figshare.com/articles/_Maze_navigation_learning_task_/680539", "title"=>"Maze navigation learning task.", "pos_in_sequence"=>0, "defined_type"=>1, "published_date"=>"2013-04-11 00:08:59"}
  • {"files"=>["https://ndownloader.figshare.com/files/1022039"], "description"=>"<p>A: The acrobot swing-up task figures a double pendulum, weakly actuated by a torque at the joint. The state of the pendulum is represented by the two angles and and the corresponding angular velocities and . The goal is to lift the tip above a certain height above the fixed axis of the pendulum, corresponding to the length of the segments. B: Goal reaching latency of TD-LTP agents. The solid line shows the median of the latencies for each trial number and the shaded area represents the 25th to 75th percentiles of the agents performance. The red line represents a near-optimal strategy, obtained by the direct search method (see <a href=\"http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1003024#s4\" target=\"_blank\">Models</a>). The blue line show the trajectory of one of the best amongst the 100 agents. The dotted line shows the limit after which a trial was interrupted if the agent did not reach the goal. C: Example trajectory of an agent successfully reaching the goal height (green line).</p>", "links"=>[], "tags"=>["neuroscience", "computational neuroscience", "Learning and memory"], "article_id"=>680540, "categories"=>["Biological Sciences"], "users"=>["Nicolas Frémaux", "Henning Sprekeler", "Wulfram Gerstner"], "doi"=>["https://dx.doi.org/10.1371/journal.pcbi.1003024.g005"], "stats"=>{"downloads"=>0, "page_views"=>0, "likes"=>0}, "figshare_url"=>"https://figshare.com/articles/_Acrobot_task_/680540", "title"=>"Acrobot task.", "pos_in_sequence"=>0, "defined_type"=>1, "published_date"=>"2013-04-11 00:09:00"}
  • {"files"=>["https://ndownloader.figshare.com/files/1022041"], "description"=>"<p>A: Cartpole swing-up problem (schematic). The cart slides on a rail of length 5, while the pole of length 1 rotates around its axis, subject to gravity. The state of the system is characterized by , while the control variable is the force exerted on the cart. The agent receives a reward proportional to the height of the pole's tip. B: Cumulative number of “successful” trials as a function of total trials. A successful trial is defined as a trial where the pole angle was maintained up () for more than 10s, out of a maximum trial length . The black line shows the median, and the shaded area represents the quartiles of 20 TD-LTP agents' performance, pooled in bins of 10 trials. The blue line shows the number of successful trials for a single agent. C: Average reward in a given trial. The average reward rate obtained during each trial is shown versus the trial number. After a rapid rise (inset, vertical axis same as main plot), the reward rises in a much slower timescale as the agents learn the finer control needed to keep the pole upright. The line and the area represent the median and the quartiles, as in B. D: Example agent behavior after 4000 trials. The three diagrams show three examples of the same agent recovering from unstable initial conditions (top: pole sideways, center: rightward speed near rail edge, bottom: small angle near rail edge).</p>", "links"=>[], "tags"=>["neuroscience", "computational neuroscience", "Learning and memory"], "article_id"=>680542, "categories"=>["Biological Sciences"], "users"=>["Nicolas Frémaux", "Henning Sprekeler", "Wulfram Gerstner"], "doi"=>["https://dx.doi.org/10.1371/journal.pcbi.1003024.g006"], "stats"=>{"downloads"=>0, "page_views"=>0, "likes"=>0}, "figshare_url"=>"https://figshare.com/articles/_Cartpole_task_/680542", "title"=>"Cartpole task.", "pos_in_sequence"=>0, "defined_type"=>1, "published_date"=>"2013-04-11 00:09:02"}
  • {"files"=>["https://ndownloader.figshare.com/files/1022042"], "description"=>"<p>A: Firing rate of rat ventral striatum “ramp cells” during a maze navigation task. In the original experiment, the rat was rewarded in two different places, first by banana flavored food pellets, corresponding to the big drop in activity, then by neutral taste food pellets, corresponding to the end of small ramp. Adapted from van der Meer and Redish <a href=\"http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1003024#pcbi.1003024-vanderMeer1\" target=\"_blank\">[44]</a>. B: Firing rate of a single critic neuron in our model from the linear track task in <a href=\"http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1003024#pcbi-1003024-g002\" target=\"_blank\">Figure 2C</a>. The dashed line indicates the firing rate (<a href=\"http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1003024#pcbi.1003024.e109\" target=\"_blank\">Eq. 12</a>) corresponding to . C: Putative network to calculate the TD error using synaptic delays. The lower right group of neurons corresponds to the critic neurons we considered in this paper. Each group of neurons gets its input delayed by the amount of the synaptic delay . Provided the synapses have the adequate efficacies (not shown), this allows the calculation of and the TD error .</p>", "links"=>[], "tags"=>["neuroscience", "computational neuroscience", "Learning and memory"], "article_id"=>680543, "categories"=>["Biological Sciences"], "users"=>["Nicolas Frémaux", "Henning Sprekeler", "Wulfram Gerstner"], "doi"=>["https://dx.doi.org/10.1371/journal.pcbi.1003024.g007"], "stats"=>{"downloads"=>0, "page_views"=>0, "likes"=>0}, "figshare_url"=>"https://figshare.com/articles/_Biological_plausibility_/680543", "title"=>"Biological plausibility.", "pos_in_sequence"=>0, "defined_type"=>1, "published_date"=>"2013-04-11 00:09:03"}
  • {"files"=>["https://ndownloader.figshare.com/files/1022043"], "description"=>"<p>A: Schematic comparison of the squared TD gradient learning rule of <a href=\"http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1003024#pcbi.1003024.e533\" target=\"_blank\">Eq. 46</a> and TD-LTP, similar to <a href=\"http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1003024#pcbi-1003024-g002\" target=\"_blank\">Figure 2A</a>. B: Linear track task using the squared TD gradient rule. Same conventions as in <a href=\"http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1003024#pcbi-1003024-g002\" target=\"_blank\">Figure 2C</a>. C: linear track task using the TD-LTP rule (reprint of <a href=\"http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1003024#pcbi-1003024-g002\" target=\"_blank\">Figure 2C</a> for comparison). D: Integrands of the disturbance term for Poisson spike train statistics. Top: squared TD gradient rule. Bottom: TD-LTP rule. In each plot the numerical value under the curve is given. This corresponds to the contribution of each presynaptic spike to the nuisance term. E: Disturbance term dependence on for the squared TD gradient rule. The mean weight change under initial conditions on an unrewarded linear track task with frozen weights, using the squared TD gradient learning rule, is plotted versus , the number of neurons composing the critic. Each cross corresponds to the mean over a 200s simulation, the plot shows crosses for each condition. The line shows a fit of the data with , the dependence form suggested by <a href=\"http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1003024#pcbi.1003024.e574\" target=\"_blank\">Eq. 50</a>. F: Same as E, for critic neurons using the TD-LTP learning rule. G, H: Same experiment as E and F, but using a rate neuron model with Gaussian noise of mean 0 and variance . The line shows a fit with , the dependence form suggested by <a href=\"http://www.ploscompbiol.org/article/info:doi/10.1371/journal.pcbi.1003024#pcbi.1003024.e574\" target=\"_blank\">Eq. 50</a>.</p>", "links"=>[], "tags"=>["neuroscience", "computational neuroscience", "Learning and memory", "nuisance"], "article_id"=>680544, "categories"=>["Biological Sciences"], "users"=>["Nicolas Frémaux", "Henning Sprekeler", "Wulfram Gerstner"], "doi"=>["https://dx.doi.org/10.1371/journal.pcbi.1003024.g008"], "stats"=>{"downloads"=>0, "page_views"=>0, "likes"=>0}, "figshare_url"=>"https://figshare.com/articles/_Alternative_learning_rule_and_nuisance_term_/680544", "title"=>"Alternative learning rule and nuisance term.", "pos_in_sequence"=>0, "defined_type"=>1, "published_date"=>"2013-04-11 00:09:04"}
  • {"files"=>["https://ndownloader.figshare.com/files/1022044"], "description"=>"<p>Numerical values of the learning rates for the different learning rules used in simulations.</p>", "links"=>[], "tags"=>["neuroscience", "computational neuroscience", "Learning and memory"], "article_id"=>680545, "categories"=>["Biological Sciences"], "users"=>["Nicolas Frémaux", "Henning Sprekeler", "Wulfram Gerstner"], "doi"=>["https://dx.doi.org/10.1371/journal.pcbi.1003024.t001"], "stats"=>{"downloads"=>0, "page_views"=>0, "likes"=>0}, "figshare_url"=>"https://figshare.com/articles/_Learning_rates_/680545", "title"=>"Learning rates.", "pos_in_sequence"=>0, "defined_type"=>3, "published_date"=>"2013-04-11 00:09:05"}

PMC Usage Stats | Further Information

  • {"unique-ip"=>"19", "full-text"=>"22", "pdf"=>"5", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"3", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2013", "month"=>"4"}
  • {"unique-ip"=>"29", "full-text"=>"29", "pdf"=>"4", "abstract"=>"1", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"4", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2013", "month"=>"5"}
  • {"unique-ip"=>"10", "full-text"=>"9", "pdf"=>"2", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"1", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2013", "month"=>"6"}
  • {"unique-ip"=>"18", "full-text"=>"16", "pdf"=>"11", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"5", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2013", "month"=>"7"}
  • {"unique-ip"=>"7", "full-text"=>"5", "pdf"=>"1", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"4", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2013", "month"=>"8"}
  • {"unique-ip"=>"9", "full-text"=>"7", "pdf"=>"1", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"7", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2013", "month"=>"9"}
  • {"unique-ip"=>"11", "full-text"=>"11", "pdf"=>"1", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"2", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2013", "month"=>"10"}
  • {"unique-ip"=>"6", "full-text"=>"8", "pdf"=>"0", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2013", "month"=>"11"}
  • {"unique-ip"=>"16", "full-text"=>"15", "pdf"=>"5", "abstract"=>"1", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"4", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2013", "month"=>"12"}
  • {"unique-ip"=>"6", "full-text"=>"6", "pdf"=>"4", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2014", "month"=>"1"}
  • {"unique-ip"=>"5", "full-text"=>"2", "pdf"=>"4", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2014", "month"=>"2"}
  • {"unique-ip"=>"8", "full-text"=>"5", "pdf"=>"5", "abstract"=>"2", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2014", "month"=>"3"}
  • {"unique-ip"=>"5", "full-text"=>"3", "pdf"=>"1", "abstract"=>"1", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"2", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2014", "month"=>"5"}
  • {"unique-ip"=>"4", "full-text"=>"3", "pdf"=>"3", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2014", "month"=>"6"}
  • {"unique-ip"=>"9", "full-text"=>"8", "pdf"=>"2", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2014", "month"=>"4"}
  • {"unique-ip"=>"11", "full-text"=>"8", "pdf"=>"4", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"1", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2014", "month"=>"7"}
  • {"unique-ip"=>"9", "full-text"=>"7", "pdf"=>"2", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"2", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2014", "month"=>"8"}
  • {"unique-ip"=>"20", "full-text"=>"11", "pdf"=>"14", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"2", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2014", "month"=>"9"}
  • {"unique-ip"=>"12", "full-text"=>"9", "pdf"=>"0", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"3", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2014", "month"=>"10"}
  • {"unique-ip"=>"13", "full-text"=>"7", "pdf"=>"2", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"8", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2014", "month"=>"11"}
  • {"unique-ip"=>"3", "full-text"=>"3", "pdf"=>"1", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2014", "month"=>"12"}
  • {"unique-ip"=>"15", "full-text"=>"9", "pdf"=>"1", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"7", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2015", "month"=>"1"}
  • {"unique-ip"=>"10", "full-text"=>"9", "pdf"=>"1", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"9", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2015", "month"=>"2"}
  • {"unique-ip"=>"6", "full-text"=>"6", "pdf"=>"5", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"1", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2015", "month"=>"3"}
  • {"unique-ip"=>"12", "full-text"=>"12", "pdf"=>"1", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"6", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2015", "month"=>"4"}
  • {"unique-ip"=>"6", "full-text"=>"6", "pdf"=>"5", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2015", "month"=>"5"}
  • {"unique-ip"=>"14", "full-text"=>"11", "pdf"=>"4", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"11", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2015", "month"=>"6"}
  • {"unique-ip"=>"17", "full-text"=>"17", "pdf"=>"1", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"1", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2015", "month"=>"7"}
  • {"unique-ip"=>"8", "full-text"=>"7", "pdf"=>"4", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"1", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2015", "month"=>"8"}
  • {"unique-ip"=>"11", "full-text"=>"13", "pdf"=>"1", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"1", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2015", "month"=>"9"}
  • {"unique-ip"=>"6", "full-text"=>"7", "pdf"=>"2", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"7", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2015", "month"=>"10"}
  • {"unique-ip"=>"13", "full-text"=>"12", "pdf"=>"4", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"2", "supp-data"=>"0", "cited-by"=>"1", "year"=>"2015", "month"=>"11"}
  • {"unique-ip"=>"5", "full-text"=>"5", "pdf"=>"1", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"2", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2015", "month"=>"12"}
  • {"unique-ip"=>"13", "full-text"=>"13", "pdf"=>"7", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"2", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2016", "month"=>"1"}
  • {"unique-ip"=>"6", "full-text"=>"6", "pdf"=>"2", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2016", "month"=>"2"}
  • {"unique-ip"=>"8", "full-text"=>"8", "pdf"=>"0", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"8", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2016", "month"=>"3"}
  • {"unique-ip"=>"21", "full-text"=>"19", "pdf"=>"5", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2016", "month"=>"4"}
  • {"unique-ip"=>"8", "full-text"=>"8", "pdf"=>"1", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"4", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2016", "month"=>"5"}
  • {"unique-ip"=>"8", "full-text"=>"7", "pdf"=>"1", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2016", "month"=>"6"}
  • {"unique-ip"=>"12", "full-text"=>"14", "pdf"=>"2", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"3", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2016", "month"=>"7"}
  • {"unique-ip"=>"7", "full-text"=>"6", "pdf"=>"3", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2016", "month"=>"8"}
  • {"unique-ip"=>"10", "full-text"=>"11", "pdf"=>"0", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"3", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2016", "month"=>"9"}
  • {"unique-ip"=>"5", "full-text"=>"4", "pdf"=>"3", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2016", "month"=>"10"}
  • {"unique-ip"=>"12", "full-text"=>"15", "pdf"=>"4", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2016", "month"=>"11"}
  • {"unique-ip"=>"10", "full-text"=>"11", "pdf"=>"2", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2016", "month"=>"12"}
  • {"unique-ip"=>"11", "full-text"=>"13", "pdf"=>"2", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2017", "month"=>"1"}
  • {"unique-ip"=>"10", "full-text"=>"11", "pdf"=>"0", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"1", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2017", "month"=>"2"}
  • {"unique-ip"=>"39", "full-text"=>"94", "pdf"=>"4", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"109", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2017", "month"=>"3"}
  • {"unique-ip"=>"11", "full-text"=>"12", "pdf"=>"1", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"2", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2017", "month"=>"4"}
  • {"unique-ip"=>"17", "full-text"=>"24", "pdf"=>"3", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"3", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2017", "month"=>"5"}
  • {"unique-ip"=>"5", "full-text"=>"7", "pdf"=>"1", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"5", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2017", "month"=>"6"}
  • {"unique-ip"=>"8", "full-text"=>"9", "pdf"=>"2", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"1", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2017", "month"=>"7"}
  • {"unique-ip"=>"17", "full-text"=>"19", "pdf"=>"3", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2017", "month"=>"8"}
  • {"unique-ip"=>"20", "full-text"=>"22", "pdf"=>"2", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"3", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2017", "month"=>"9"}
  • {"unique-ip"=>"20", "full-text"=>"22", "pdf"=>"5", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2017", "month"=>"10"}
  • {"unique-ip"=>"18", "full-text"=>"17", "pdf"=>"1", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"2", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2017", "month"=>"11"}
  • {"unique-ip"=>"15", "full-text"=>"17", "pdf"=>"3", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2017", "month"=>"12"}
  • {"unique-ip"=>"17", "full-text"=>"20", "pdf"=>"6", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"7", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2018", "month"=>"1"}
  • {"unique-ip"=>"12", "full-text"=>"12", "pdf"=>"5", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"8", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2018", "month"=>"3"}
  • {"unique-ip"=>"15", "full-text"=>"13", "pdf"=>"4", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"1", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2019", "month"=>"1"}
  • {"unique-ip"=>"17", "full-text"=>"18", "pdf"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"4", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2018", "month"=>"4"}
  • {"unique-ip"=>"14", "full-text"=>"15", "pdf"=>"3", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2018", "month"=>"5"}
  • {"unique-ip"=>"25", "full-text"=>"29", "pdf"=>"6", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2018", "month"=>"6"}
  • {"unique-ip"=>"9", "full-text"=>"8", "pdf"=>"2", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2018", "month"=>"7"}
  • {"unique-ip"=>"12", "full-text"=>"12", "pdf"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2018", "month"=>"8"}
  • {"unique-ip"=>"12", "full-text"=>"15", "pdf"=>"2", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2018", "month"=>"9"}
  • {"unique-ip"=>"13", "full-text"=>"19", "pdf"=>"1", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2018", "month"=>"10"}
  • {"unique-ip"=>"23", "full-text"=>"22", "pdf"=>"4", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"8", "supp-data"=>"0", "cited-by"=>"1", "year"=>"2018", "month"=>"11"}
  • {"unique-ip"=>"18", "full-text"=>"19", "pdf"=>"5", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2018", "month"=>"12"}
  • {"unique-ip"=>"12", "full-text"=>"11", "pdf"=>"1", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"5", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2019", "month"=>"2"}
  • {"unique-ip"=>"7", "full-text"=>"6", "pdf"=>"1", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2019", "month"=>"3"}
  • {"unique-ip"=>"23", "full-text"=>"20", "pdf"=>"7", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"8", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2019", "month"=>"4"}
  • {"unique-ip"=>"6", "full-text"=>"5", "pdf"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"6", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2019", "month"=>"5"}
  • {"unique-ip"=>"9", "full-text"=>"9", "pdf"=>"2", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"3", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2019", "month"=>"8"}
  • {"unique-ip"=>"16", "full-text"=>"12", "pdf"=>"6", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2019", "month"=>"9"}
  • {"unique-ip"=>"13", "full-text"=>"14", "pdf"=>"3", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2019", "month"=>"10"}
  • {"unique-ip"=>"17", "full-text"=>"12", "pdf"=>"8", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2019", "month"=>"12"}
  • {"unique-ip"=>"26", "full-text"=>"22", "pdf"=>"12", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"1", "supp-data"=>"0", "cited-by"=>"2", "year"=>"2020", "month"=>"2"}
  • {"unique-ip"=>"34", "full-text"=>"33", "pdf"=>"10", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"2", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2020", "month"=>"3"}
  • {"unique-ip"=>"18", "full-text"=>"20", "pdf"=>"4", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2020", "month"=>"4"}
  • {"unique-ip"=>"37", "full-text"=>"33", "pdf"=>"15", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2020", "month"=>"5"}
  • {"unique-ip"=>"17", "full-text"=>"18", "pdf"=>"5", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2020", "month"=>"6"}
  • {"unique-ip"=>"7", "full-text"=>"5", "pdf"=>"3", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2020", "month"=>"7"}
  • {"unique-ip"=>"9", "full-text"=>"8", "pdf"=>"2", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2020", "month"=>"8"}
  • {"unique-ip"=>"6", "full-text"=>"7", "pdf"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2020", "month"=>"9"}
  • {"unique-ip"=>"15", "full-text"=>"14", "pdf"=>"1", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2020", "month"=>"10"}

Relative Metric

{"start_date"=>"2013-01-01T00:00:00Z", "end_date"=>"2013-12-31T00:00:00Z", "subject_areas"=>[{"subject_area"=>"/Biology and life sciences", "average_usage"=>[269, 466, 588, 697, 800, 896, 988, 1076, 1165, 1254, 1340, 1417]}, {"subject_area"=>"/Biology and life sciences/Anatomy", "average_usage"=>[248, 440, 562, 666, 765, 856, 944, 1028, 1112, 1198, 1282, 1362, 1430]}, {"subject_area"=>"/Biology and life sciences/Anatomy and physiology", "average_usage"=>[256, 428]}, {"subject_area"=>"/Biology and life sciences/Cell biology", "average_usage"=>[272, 472, 600, 713, 815, 911, 1004, 1094, 1185, 1273, 1358, 1441]}, {"subject_area"=>"/Biology and life sciences/Psychology", "average_usage"=>[284, 458, 579, 688, 782, 879, 957, 1048, 1126, 1204, 1280, 1357, 1415]}, {"subject_area"=>"/Computer and information sciences/Neural networks", "average_usage"=>[284, 481, 590, 705, 782, 880, 969, 1051, 1141, 1227, 1337, 1399, 1463]}, {"subject_area"=>"/Medicine and health sciences", "average_usage"=>[264, 460, 584, 692, 794, 887, 978, 1067, 1154, 1241, 1328, 1408, 1474]}, {"subject_area"=>"/Medicine and health sciences/Anatomy", "average_usage"=>[248, 441, 564, 668, 769, 859, 948, 1034, 1117, 1202, 1290, 1368, 1437]}, {"subject_area"=>"/Medicine and health sciences/Anatomy and physiology", "average_usage"=>[253, 430]}, {"subject_area"=>"/Physical sciences/Chemistry", "average_usage"=>[247, 429, 544, 647, 747, 842, 929, 1012, 1099, 1179, 1263, 1339, 1409]}]}
Loading … Spinner
There are currently no alerts