limbra issueshttps://gitlab.in2p3.fr/groups/limbra/-/issues2021-04-30T14:56:00+02:00https://gitlab.in2p3.fr/limbra/limbra/-/issues/96Migrate to new inspirehep API2021-04-30T14:56:00+02:00LE GAC RenaudMigrate to new inspirehep API* Since March 2020 a new JSON API is available to search and to retrieve publications from `inspirehep.net`
* The old JSON API is available from `old.inspirehep.net`. It is also used by `cds.cern.ch`.
* `Inspirehep` will close the old AP...* Since March 2020 a new JSON API is available to search and to retrieve publications from `inspirehep.net`
* The old JSON API is available from `old.inspirehep.net`. It is also used by `cds.cern.ch`.
* `Inspirehep` will close the old API in the coming month.
* As a consequence, LIMBRA will have to deal with two stores (cds and inspirehep) working with a different API
* Documentation for the new API: https://github.com/inspirehep/rest-api-dochttps://gitlab.in2p3.fr/limbra/limbra/-/issues/95old.inspirehep.net2020-04-17T15:21:52+02:00LE GAC Renaudold.inspirehep.net* 30 March 2020 a new version of `inspirehep.net` was put in production
* For publications, the API is not yet ready (user restrictions). Users have to use `old.inspirehep.net`
* For conferences, the new API is open: https://inpsirehep.n...* 30 March 2020 a new version of `inspirehep.net` was put in production
* For publications, the API is not yet ready (user restrictions). Users have to use `old.inspirehep.net`
* For conferences, the new API is open: https://inpsirehep.net/api/conference/1754632
* For institutions, use old.inspirehep.net
* The syntax for new API is based on elastic search. Quite different from the previous one
* Resit methods of `InvenioStore` `get_ids`, `get_record` and `load_record`
- Introduce the concept of **shelf**: `None`, `literature`, `conferences` and `institutions`
- keep two hosts `inspirehep.net` and `cds.cern.ch`.
- define properly the URL in `get_ids` and `get_record` according to the host and to the shelf.
* Propage change in Automaton and controllers (see [InvenioStore.pdf](/uploads/794a52bcdfa502b42e3f601d8c33651c/InvenioStore.pdf))
Some references:
* https://invenio.readthedocs.io/en/latest/quickstart/crud-operations.html#search-for-records
* https://invenio-records-rest.readthedocs.io/en/latest/usage.html
https://gitlab.in2p3.fr/limbra/limbra/-/issues/94migrate to python 3.72019-12-05T18:23:20+01:00LE GAC Renaudmigrate to python 3.7* require web2py 2.18.3 or above
* require plugin_dbui 0.9.9.1-py37 or above* require web2py 2.18.3 or above
* require plugin_dbui 0.9.9.1-py37 or abovehttps://gitlab.in2p3.fr/limbra/limbra/-/issues/93fix bugs2019-10-31T10:11:03+01:00LE GAC Renaudfix bugs* [x] the role `user` is not working any more.
* [x] The affiliation key `Ecole Polytechnique` select autors from LLR (`Ecole Polytechnique`) and EPFL (`Ecole Polytechnique, Lausanne`)
* [x] Internal server error https://inspirehep.net/r...* [x] the role `user` is not working any more.
* [x] The affiliation key `Ecole Polytechnique` select autors from LLR (`Ecole Polytechnique`) and EPFL (`Ecole Polytechnique, Lausanne`)
* [x] Internal server error https://inspirehep.net/record/1713704 (article notice error)
* [x] Internal server error https://inspirehep.net/record/1692891 (proceeding missing 909CO field))
* [x] Internal server error https://inspirehep.net/record/1610670 (proceeding missing 909CO field)
* [x] Add a button Go in the selector of the publication tables
* [x] Allow user to modify authors rolehttps://gitlab.in2p3.fr/limbra/limbra/-/issues/92Disable mutiple collection in the field harvesters.collections2018-04-27T15:02:17+02:00LE GAC RenaudDisable mutiple collection in the field harvesters.collections* The database field collections `harvesters.collections` can contain several collections separated by a comma,*e.g.* `LHCb Papers, LHCb Detector Performance Papers`
* This behavior is not compatible with collection defined using author ...* The database field collections `harvesters.collections` can contain several collections separated by a comma,*e.g.* `LHCb Papers, LHCb Detector Performance Papers`
* This behavior is not compatible with collection defined using author name, *e.g.* `find a nguyen, v g`.
* Modify the logic to one harvesters one collection
* Create a script `fixHarvestersCollections.py` to fix multiple cds collections.https://gitlab.in2p3.fr/limbra/limbra/-/issues/91Increase the length of the database field harvesters.collection2018-04-27T15:02:17+02:00LE GAC RenaudIncrease the length of the database field harvesters.collection* the lenght of the field `harvesters.collections` is `255`
* It is too small to deal with a list of theorists at IPNO
* Increase it to `1024`
* Create a script `fixHarvesterscollection.py` to change it in all database.* the lenght of the field `harvesters.collections` is `255`
* It is too small to deal with a list of theorists at IPNO
* Increase it to `1024`
* Create a script `fixHarvesterscollection.py` to change it in all database.https://gitlab.in2p3.fr/limbra/limbra/-/issues/90improve harvester wizard and documentation when dealing with author2018-04-27T15:02:17+02:00LE GAC Renaudimprove harvester wizard and documentation when dealing with author* Rather tricky to parameterise
* I suspect a bug in the documentation `find (a j cronin or v fitch)`. The correct syntax is `find a j cronin or a v fitch`.
* The correct way is to alway specify the keyword `a` and to not use parenthes...* Rather tricky to parameterise
* I suspect a bug in the documentation `find (a j cronin or v fitch)`. The correct syntax is `find a j cronin or a v fitch`.
* The correct way is to alway specify the keyword `a` and to not use parenthesis.
* In addition author names can be written `j cronin` or `cronin, j`.
* For author with a double first name, there is only one way to write them `nguyen, v g`
* For author with a first name starting with the letter `a` , we shall use `find a albert einstein`
* A working example is the theory group from ipno. The following filter find their proceedings for 2016, but I don't know if all of them are found?:
```
find a h sazdjian or a lansberg, j p or a s friot or a b moussallam or a v bernard or a j carbonell and date 2016 and tc c
```
To fix this issue:
* [ ] modify the documentation
* [ ] modify the wizard to create harvester
* [ ] modify the script to parameterize an institute `deploy.py`https://gitlab.in2p3.fr/limbra/limbra/-/issues/87Remove the password field in user form2018-04-27T15:02:17+02:00LE GAC RenaudRemove the password field in user form* `user` table
* With the form is is possible to change the password.
* This action should not be allowed since the password is encrypted by the server.
* Unfortunatetly, the encryption is not performed when using the form
* Hide this f...* `user` table
* With the form is is possible to change the password.
* This action should not be allowed since the password is encrypted by the server.
* Unfortunatetly, the encryption is not performed when using the form
* Hide this fieldhttps://gitlab.in2p3.fr/limbra/limbra/-/issues/86Use pandas.DataFrame in record for author and their affiliation2018-04-27T15:02:17+02:00LE GAC RenaudUse pandas.DataFrame in record for author and their affiliation* Class `invenio_tools.RecordPubli`
* Store the authors and their affiliation in a DataFrame with two columns `raw_author` and `affiliation`
* Columns like `format_author`, `first_name`, `last_name` can be added when required.
* Vector...* Class `invenio_tools.RecordPubli`
* Store the authors and their affiliation in a DataFrame with two columns `raw_author` and `affiliation`
* Columns like `format_author`, `first_name`, `last_name` can be added when required.
* Vectorize the function `format_author_fr` and `familly_name_fr`.
* Should speed up the author / affilation processing and simplify the code.https://gitlab.in2p3.fr/limbra/limbra/-/issues/85Order in conference in list2018-04-27T15:02:17+02:00LE GAC RenaudOrder in conference in list* CPPM
* List of COM for the LHCb and for the year 2016 (2017)
* The order should be defined by the date of the conference
* Few mistakes can be observed !* CPPM
* List of COM for the LHCb and for the year 2016 (2017)
* The order should be defined by the date of the conference
* Few mistakes can be observed !https://gitlab.in2p3.fr/limbra/limbra/-/issues/84Fail to insert MarcXML2018-04-27T15:02:17+02:00LE GAC RenaudFail to insert MarcXML* `LHCb-ANA-2016-056`
* `https://cds.cern.ch/record/2200251`
* Failed to insert the MarcXML → invalid host
* In addition, the action `edit & insert` is failing since the store is not defined.
* The fix should probably be to add the store...* `LHCb-ANA-2016-056`
* `https://cds.cern.ch/record/2200251`
* Failed to insert the MarcXML → invalid host
* In addition, the action `edit & insert` is failing since the store is not defined.
* The fix should probably be to add the store in the MarcXML selector.....https://gitlab.in2p3.fr/limbra/limbra/-/issues/83Speed up graph2018-04-27T15:02:17+02:00LE GAC RenaudSpeed up graph* deleguate dirty task to `DataFrame` instead of the database and python functions.* deleguate dirty task to `DataFrame` instead of the database and python functions.https://gitlab.in2p3.fr/limbra/limbra/-/issues/82Speed up the model2018-04-27T15:02:17+02:00LE GAC RenaudSpeed up the model* Follow the `tev/plugin_event` implementaion
* Create the module `auth` with the function `configure_auth`
* Create modules `model_core`, `model_report`, `model_selector` with the class `Core`, `Report` and `Selector`
* Create modules ...* Follow the `tev/plugin_event` implementaion
* Create the module `auth` with the function `configure_auth`
* Create modules `model_core`, `model_report`, `model_selector` with the class `Core`, `Report` and `Selector`
* Create modules `ui_core`, `ui_report`, `ui_selector` and `ui_viewport` with the classes `CoreUi`, `ReportUi`, `SelectorUi` and `ViewportUi`
* Reduce the model to one short file `main.py`
The main steps are:
- [x] models
- [x] lazyT
- [x] use `current.db`, `current.virtdb`, `current.auth`
- [x] replace `'` by `"`
- [x] run `pylint`https://gitlab.in2p3.fr/limbra/limbra/-/issues/81Migrate to plugin_dbui 0.9.8.12018-04-27T15:02:17+02:00LE GAC RenaudMigrate to plugin_dbui 0.9.8.1* The release 0.9.8.1 is not backward compatible.* The release 0.9.8.1 is not backward compatible.https://gitlab.in2p3.fr/limbra/limbra/-/issues/79Resurect the standalone url in lists and metrics views2018-04-27T15:02:17+02:00LE GAC RenaudResurect the standalone url in lists and metrics views* Le list views as well as the metrics have a standalone URL.
* With the commit dd6e98f8d0, it should work again for the list. This is true in the `dev` environment but not on the server `marprod`.
* For the metrics, the URL is not wor...* Le list views as well as the metrics have a standalone URL.
* With the commit dd6e98f8d0, it should work again for the list. This is true in the `dev` environment but not on the server `marprod`.
* For the metrics, the URL is not working since the JavaScript librairy `Ext JS` is missing.
* A possible way out is to add a new parameter in the selector, *e.g* `standalone`with the default value`false`.
* The standalone parameter is set to `true` when using the standalone URL.
* The controller `list/index` is the almost the same. the only change is the selection of the view which depends on the `standalone` value: ` either `lists/index.html` or `lists/standalone.html`.
* The html page `standalone` contains the header and the body. In the header contains the `Ext JS` and the `MathJax` libraries. The body include the `index.html` view.
* The default value can be set in existing list and metric by adding `baseParams={'standalone': False}` when building the lists/metrics node in the main viewport.
TODO:
* [ ] modify the controller `lists/index.html`
* [ ] add the view `views/lists/standalone.html`
* [ ] modify the node `lists` in `viewport`
* [ ] propagate changes to `metrics` and even to `graphs`https://gitlab.in2p3.fr/limbra/limbra/-/issues/78Add the number of citations2020-03-20T17:30:27+01:00LE GAC RenaudAdd the number of citations* It is now possible to collect the *number of citations*:
- http://inspirehep.net/record/1471725?of=recjson&ot=number_of_citations.
- https://inspirehep.net/search?recid=1762038&ot=number_of_citations&of=recjson
* Add a databa...* It is now possible to collect the *number of citations*:
- http://inspirehep.net/record/1471725?of=recjson&ot=number_of_citations.
- https://inspirehep.net/search?recid=1762038&ot=number_of_citations&of=recjson
* Add a database table, `citations`, containing citations per publications as a function of time:
date,
publications.id,
number_of_citations
* Add a web2py cron job updating the number of citation once a week (http://web2py.com/books/default/chapter/29/04/the-core#Cron)
* Build a pdf report contains plots (article):
- `number of publications` as a function of the `number of citations`
- `box plot` on the number of citation all, per domaine scientifique, per team, or per project
https://gitlab.in2p3.fr/limbra/limbra/-/issues/77Migrate from MARC to JSON format2020-02-12T16:48:07+01:00LE GAC RenaudMigrate from MARC to JSON format* Stores `inspirehep.net` and `cds.cern.ch` can return a JSON string instead of an XML one.
* For example: http://inspirehep.net/record/1471725?of=recjson
* More on the JSON API: https://inspirehep.net/info/hep/api?ln=fr
* Map JSON / ...* Stores `inspirehep.net` and `cds.cern.ch` can return a JSON string instead of an XML one.
* For example: http://inspirehep.net/record/1471725?of=recjson
* More on the JSON API: https://inspirehep.net/info/hep/api?ln=fr
* Map JSON / MARC: https://github.com/inspirehep/invenio/blob/prod/modules/bibfield/etc/atlantis.cfg
* Comparison between JSON and MARC in [CodesJsonMarc.pdf](/uploads/644fd6af927f05d54064baba7fb5b7ab/CodesJsonMarc.pdf) and [Json_fields.pdf](/uploads/6d30afc3ff85bbc7389dcf5d81159f01/Json_fields.pdf)https://gitlab.in2p3.fr/limbra/limbra/-/issues/73Harvest the Hall Hidden collection (inspirehep.net)2018-04-27T15:02:17+02:00LE GAC RenaudHarvest the Hall Hidden collection (inspirehep.net)https://gitlab.in2p3.fr/limbra/limbra/-/issues/72Add author condition in the selector for metrics and graphs2018-04-27T15:02:17+02:00LE GAC RenaudAdd author condition in the selector for metrics and graphs* Same set of condition for lists, metrics and graphs
* Add the authors fields.
* The comparator operator is `contains`
* ...* Same set of condition for lists, metrics and graphs
* Add the authors fields.
* The comparator operator is `contains`
* ...https://gitlab.in2p3.fr/limbra/limbra/-/issues/71Migrate to Ext JS 6.0.12018-04-27T15:02:17+02:00LE GAC RenaudMigrate to Ext JS 6.0.1