Commit a50cc714 authored by LE GAC Renaud's avatar LE GAC Renaud
Browse files

Merge branch 'master' into 'production'

Release 0.9.7.0

See merge request !93
parents 78eee94f 60fa303d
......@@ -2,6 +2,18 @@
HEAD
0.9.7.0 (Oct 2017)
- Require plugin_dbui 0.9.8.3 or higher.
- Increase the length of the field harvesters.collections to 1024
instead of 255.
- Change the rule for the field harvesters.collections. It now contains
the definiton for a single collection (it was possible to define several
collection separated by a comma but this confict with author encoding).
- Fix several bugs in the documentation and in the wizard harvesters.
They are related to the defintion of a collection using authors.
- Provide two scripts to change the harvesters.collections behavhior
for the existing database.
0.9.6.5 (Jun 2017)
- Require plugin_dbui 0.9.8.2 or higher.
- Factorize the models in classes.
......
0.9.6.5
\ No newline at end of file
0.9.7.0
\ No newline at end of file
......@@ -350,13 +350,14 @@ def harvester():
collection.append("cn %s" % collaboration)
else:
collection.append("(a %s)" % " or ".join(rvars.people))
authors = ["a %s" % elt for elt in rvars.people]
collection.append(" or ".join(authors))
if automaton == "articles":
collection.append("tc p and not tc c")
elif automaton == "proceedings":
collection.append("tc p and tc c")
collection.append("tc c")
values.collections = "find %s" % " and ".join(collection)
......
......@@ -53,11 +53,6 @@ ALEPH, ATLAS, ALICE, CMS, ...
La collaboration LHCb definit la collection *LHCb Papers* et la collection
*LHCb Detector Performance Papers*. La première contient tous les articles
décrivant les résultats de physique, et la seconde ceux
décrivant les performances du détecteur. Il est possible de moissoner ces deux
collections avec un seul moissonneur. Pour celà, Il suffit de remplir
le champ *collections* du formulaire de la :numref:`fig-form-harvester`
avec le nom des collections séparées par une virgule :
.. line-block::
LHCb Papers, LHCb Detector Performance Papers
décrivant les performances du détecteur. Il faut donc créer deux moissonneurs,
un par collection, pour scanner tous les articles publiés par la collaboration
LHCb.
......@@ -17,8 +17,8 @@ moissonnables :
* articles
* actes de conférence
Une collection est définit par les *mots clés* qui sont utilisés par le moteur
de recherche de *inspirehep.net* :
Une collection est définit par les *mots clés* qui sont utilisés par le `moteur
de recherche <http://inspirehep.net/info/hep/search-tips>`_ de inspirehep.net :
#. Le premier terme est toujours :
......@@ -45,7 +45,14 @@ de recherche de *inspirehep.net* :
.. line-block::
``and (a j cronin or v fitch)``
``and a j cronin or a v fitch``
.. note::
Le nom de l'auteur doir toujours être pécédé du mot-clé `a`
.. note::
Quand le prénon de l'auteur commence par un *a*, il faut le
spécifier en enitier, *i.e.* `a albert einstein`
#. Le troisième terme définit le type de la publication :
......@@ -67,7 +74,7 @@ deux cas de figures évoqué ci-dessus, sont les suivantes :
.. line-block::
``find cn lhcb and tc p and not tc c``
``find (a j cronin or v fitch) and tc p and not tc c``
``find a j cronin or a v fitch and tc p and not tc c``
Les définitions des collections pour l'expérience LHCb sont données dans la
:numref:`table-collection-inspirehep`.
......
......@@ -47,8 +47,8 @@ Il a cinq champs :
* **entrepôt**
choisir un entrepôt permis les valeurs proposées.
* **collections**
ce champ définit la(es) collection(s) de l'entrepôt qui vont être
scannée(s). La définition d'une collection dépend de l'entrepôt
ce champ définit la collection de l'entrepôt qui va être
scannée. La définition d'une collection dépend de l'entrepôt
et de la catégorie des notices.
.. important::
......
......@@ -684,6 +684,7 @@
'The "query" is a condition like "db.table1.field1==\'value\'". Something like "db.table1.field1==db.table2.field2" results in a SQL JOIN.': 'The "query" is a condition like "db.table1.field1==\'value\'". Something like "db.table1.field1==db.table2.field2" results in a SQL JOIN.',
'The <i>record id</i> is not well formed.': 'Le <i>record id</i> est mal formé.',
'The category is used twice !': 'La catégorie est utilisée deux fois !',
"the collection to be interrogated, e.g. 'LHCb Papers' or 'find cn lhcb and tc p and not tc c'.": "la collection a interroger, i.e. 'LHCb Papers' ou 'find cn lhcb and tc p and not tc c'.",
'The field "%s" is missing ...': 'Le champ "%s" est manquant ...',
"The field '%s' is missing ...": "The field '%s' is missing ...",
'The generation of the ODT file failed:<br><br>': 'La génération du fichier ODT a échoué :<br><br>',
......
......@@ -483,23 +483,31 @@ class Automaton(object):
self.collection_logs.append(MsgCollection(found=1))
self.decode_xml(xml)
def process_url(self, host, collections):
def process_url(self, host, collection):
"""Retrieve the XML string from the invenio store and
insert corresponding records in the database.
Args:
host (unicode): host name to query for publications, either
host (str):
host name to query for publications, either
``cds.cern.ch`` or ``inspirehep.net``.
collections (unicode): list of collection to be interrogated.
collection (str):
the collection to be interrogated.
Raises:
StoreException: when something goes wrong interrogating the
store.
Marc12Exception: when something goes wrong decoding the XML
string return by the store.
CheckException: when the record has non-conformities.
Exception: when the python code crashes.
StoreException:
when something goes wrong interrogating the store.
Marc12Exception:
when something goes wrong decoding the XML string
return by the store.
CheckException:
when the record has non-conformities.
Exception:
when the python code crashes.
"""
if self.dbg:
......@@ -507,13 +515,10 @@ class Automaton(object):
# extend harvester for logs
self.harvester.host = host
self.harvester.collections = collections
self.harvester.collection = collection
store = InvenioStore(host)
# list of collections
collections = re.sub(" *, *", ",", collections).split(",")
# alias
collection_logs = self.collection_logs
controller = self.controller
......@@ -522,63 +527,61 @@ class Automaton(object):
logs = self.logs
project = self.db.projects[self.id_project].project
# extract the list of publications from the store for each collection
# extract the list of publications from the store for the collection
# the search is perform on a range of creation date
# if not defined all element are return
#
# The method uses here minimise the memory usage
# on the server as well as on the client side
for collection in collections:
# log collection information
# A collection is identified as "Project Controller collection"
title = "%s / %s / %s" % (project, controller, collection)
collection_logs.append(MsgCollection(title=title))
# log collection information
# A collection is identified as "Project Controller collection"
title = "%s / %s / %s" % (project, controller, collection)
collection_logs.append(MsgCollection(title=title))
# search record in the harvester repository
kwargs = self._search_parameters(collection)
# search record in the harvester repository
kwargs = self._search_parameters(collection)
try:
rec_ids = store.get_ids(**kwargs)
try:
rec_ids = store.get_ids(**kwargs)
except Exception as error:
collection_logs[-1].url = store.last_search_url()
collection_logs[-1].error = error
return
except Exception as error:
collection_logs[-1].url = store.last_search_url()
collection_logs[-1].error = error
continue
collection_logs[-1].url = store.last_search_url()
collection_logs[-1].found = len(rec_ids)
collection_logs[-1].url = store.last_search_url()
collection_logs[-1].found = len(rec_ids)
if not rec_ids:
return
if not rec_ids:
continue
if self.dbg:
print "%i records found in %s" % (len(rec_ids), collection)
if self.dbg:
print "%i records found in %s" % (len(rec_ids), collection)
for rec_id in rec_ids:
for rec_id in rec_ids:
if self.dbg:
print "\nprocessing record", rec_id
if self.dbg:
print "\nprocessing record", rec_id
try:
db_id = is_record_in_db(title, host=host, rec_id=rec_id)
if db_id:
if self.dbg:
print "record in db", rec_id, "->", db_id
continue
xml = store.get_record(rec_id)
decode_xml(xml)
except Exception as e:
print traceback.format_exc()
url = OAI_URL % (host, rec_id)
logs.append(Msg(harvester=self.harvester,
collection=title,
record_id=rec_id,
title=url))
logs[-1].reject(e)
try:
db_id = is_record_in_db(title, host=host, rec_id=rec_id)
if db_id:
if self.dbg:
print "record in db", rec_id, "->", db_id
return
xml = store.get_record(rec_id)
decode_xml(xml)
except Exception as e:
print traceback.format_exc()
url = OAI_URL % (host, rec_id)
logs.append(Msg(harvester=self.harvester,
collection=title,
record_id=rec_id,
title=url))
logs[-1].reject(e)
def decode_xml(self, xml):
"""Decode the MARC XML string and insert records in the database.
......
......@@ -117,7 +117,7 @@ class CheckAndFix(object):
iterselect = db(db.affiliation_keys.id > 0).iterselect()
reg_institute = \
u"|".join(imap(
lambda row: u"^%(key_u)s%(key_v)s" % row, iterselect))
lambda row: u"%(key_u)s%(key_v)s" % row, iterselect))
# protect special character
reg_institute = (reg_institute
......
......@@ -37,7 +37,8 @@ MSG_INVALID_FMT = "Invalid format for author"
# The MARC12 keys containing paper reference
PAPER_REFERENCE_KEYS = set(["c", "p", "v", "y"])
REG_INITIAL = initial = r"^(\w+)\.?(\-)* *(\w+)*\.?$"
# Limit the number of first name to two (others will be ignored)
REG_INITIAL = initial = r"^(\w+)\.?(\-)* *(\w+)*\.?"
def to_initial(x, y, z):
......
......@@ -150,8 +150,8 @@ class Harvester(object):
"""
tp_collections = \
T("List of collections separated by comma: "
"LHCb Papers, LHCb Talks")
T("the collection to be interrogated, e.g. "
"'LHCb Papers' or 'find cn lhcb and tc p and not tc c'.")
tp_host = \
T("Address of the invenio store where the search is performed.")
......@@ -184,7 +184,7 @@ class Harvester(object):
Field("collections",
"string",
length=255,
length=1024,
comment=tp_collections,
notnull=True),
......
......@@ -2,7 +2,7 @@
"""User Interface for the authentication tables
"""
from plugin_dbui import GridModifier
from plugin_dbui import FormModifier, GridModifier
class AuthUI(object):
......@@ -62,6 +62,13 @@ class AuthUI(object):
T (gluon.languages.translator): language translator
"""
# ....................................................................
#
# Form
#
mdf = FormModifier("auth_user")
mdf.hide_fields("password")
# ....................................................................
#
# Grid
......
......@@ -2,6 +2,7 @@
"""User Interface for the harvester tables
"""
from gluon import current
from plugin_dbui import (FieldsModifier,
FormModifier,
GridModifier,
......@@ -97,7 +98,7 @@ class HarvesterUI(object):
# Fields
#
mdf = FieldsModifier("harvesters")
mdf.configure_field("collections", xtype="textarea")
mdf.configure_field("collections", height=250, xtype="textarea")
mdf.configure_field("ratio", hidden=True)
mdf.configure_field("scan", hidden=True)
......@@ -147,8 +148,11 @@ class HarvesterUI(object):
mdf.configure_filters(plugins=["pFormToolTip"], width=300)
mdf.configure_gridWithFilter(selectorTitle=T("Filter"))
mdf.configure(plugins=[{"ptype": "pGridRowEditorAddWizard",
"wizard": {"xtype": "xwizardharvester"}}])
mdf.configure(plugins=[{
"ptype": "pGridRowEditorAddWizard",
"wizard": {
"url": "/%s/wizards/harvester" % current.request.application,
"xtype": "xwizardharvester"}}])
# ....................................................................
#
......
......@@ -352,7 +352,7 @@ CREATE TABLE `harvesters` (
`id_projects` int(11) DEFAULT NULL,
`controller` varchar(255) NOT NULL,
`host` varchar(255) NOT NULL DEFAULT 'cdsweb.cern.ch',
`collections` varchar(255) DEFAULT NULL,
`collections` varchar(1024) DEFAULT NULL,
`id_categories` int(11) DEFAULT NULL,
`scan` char(1) DEFAULT NULL,
PRIMARY KEY (`id`),
......
collaboration,synonyms
ANTARES Collaboration,|ANTARES|
ATLAS Collaboration,|ATLAS|
D0 Collaboration,|D0|
"CDF, D0 Collaborations",|CDF and D0 Collaboration|
H1 Collaboration,"|H1 , Zeus| H1 and ZEUS Collaborations|H1|ZEUS|"
LHCb Collaboration,|LHCb|
SDSS Collaboration,|SDSS|
BOSS Collaboration,<NULL>
CTA Consortium Collaboration,|CTA|CTA Collaboration|CTA Consortium|
"H1, ZEUS Collaborations","|H1 and ZEUS Collaboration|ZEUS Collaboration, H1 Collaboration| H1 and ZEUS Collaborations|"
Euclid Collaboration,<NULL>
"ANTARES, TAROT Collaborations","|ANTARES Collaboration and TAROT Collaboration|ANTARES Collaboration, TAROT Collaboration|"
LSST Dark Energy Science Collaboration,||
"ALEPH Collaboration, DELPHI Collaboration, L3 Collaboration, OPAL Collaboration, LEP Electroweak Working Group",||
ALEPH Collaboration,|ALEPH|
"ANTARES, ROTSE, TAROT Collaborations",||
ATLAS LArg Collaboration,|ATLAS Liquid Argon Calorimeter Collaboration|
KM3NeT Collaboration,|KM3NeT|
"ANTARES, LIGO, VIRGO Collaborations",||
NA48/2 Collaboration,||
Heavy Flavor Averaging Group,||
ATLAS R&D,||
"ATLAS, CMS Collaborations","|CMS Collaboration, ATLAS Collaboration|ATLAS and CMS Collaborations|"
NEMO-3 Collaboration,||
SNLS Collaboration,||
SNFactory Collaboration,||
ATLAS TDAQ Collaboration,||
"CMS, LHCb Collaborations",|CMS Collaboration & LHCb Collaboration|CMS and LHCb Collaboration|
"ANTARES, TANAMI Collaborations",|TANAMI and ANTARES Collaborations|
"ANTARES, IceCube Collaborations","|ANTARES, IceCube|"
"ANTARES, ROTSE, TAROT, Swift, Zadko Collaborations","|ZADKO Collaboration, SWIFT Collaboration | ANTARES, ROTSE, TAROT, SWIFT, ZADKO| Antares, Rotse, Tarot, Swift, Zadko Collaborations| ANTARES, ROTSE, TAROT, Swift, Zadko|"
"ANTARES, IceCube, LIGO, VIRGO Collaborations","|ANTARES Collaboration, IceCube Collaboration, LIGO Scientific Collaboration, VIRGO Collaboration|ANTARES and IceCube and LIGO Scientific and Virgo Collaborations|"
"ANTARES, TAROT, ROTSE, Swift, Zadko Collaborations","|ANTARES, TAROT, ROTSE, Swift, Zadko|"
"MWA, ANTARES, TAROT, ROTSE Collaborations","|MWA, ANTARES,TAROT, ROTSE|"
DESI Collaboration,||
HESS Collaboration,"|HESS|H.E.S.S and LAT Collaborations|H.E.S.S.|H.E.S.S. and Fermi-LAT|MAGIC, HESS|"
"ANTARES, ROTSE, TAROT, MASTER, Swift Collaborations","|ZADKO Collaboration, SWIFT Collaboration|ANTARES, ROTSE, TAROT, SWIFT, MASTER, ZADKO|Antares, Rotse, Tarot, Swift, Zadko Collaborations|ANTARES, ROTSE, TAROT, Swift, Zadko|"
SVOM Collaboration,|SVOM|
"ANTARES, HESS Collaborations",|HESS and ANTARES Collaborations|
collaboration,controller,host,collection,comment
a1,articles,inspirehep.net,find cn a1 and tc p and not tc c,mami accelerator
auger,articles,inspirehep.net,find cn auger and tc p and not tc c,auger or pierre auger give the same result
aleph,articles,cds.cern.ch,ALEPH Papers,
alice,articles,cds.cern.ch,ALICE Papers,
ams,articles,inspirehep.net,find cn ams and tc p and not tc c,
antares,articles,inspirehep.net,find cn antares and tc p and not tc c,
athena,articles,inspirehep.net,find athena and not cn athena and tc p and not tc c,warning: détecteur X embarqué ESA not CERN coll.
atlas,articles,cds.cern.ch,ATLAS Papers,
babar,articles,inspirehep.net,find cn babar and tc p and not tc c,
belle,articles,inspirehep.net,find cn belle and tc p and not tc c,
belle-ii,articles,inspirehep.net,find cn belle-ii and tc p and not tc c,
borexino,articles,inspirehep.net,find cn borexino and tc p and not tc c,
boss,articles,inspirehep.net,find cn boss and tc p and not tc c,
calice,articles,inspirehep.net,find cn calice and tc p and not tc c,
celeste,articles,inspirehep.net,find cn celeste and tc p and not tc c,
ckmfitter,articles,inspirehep.net,find ckmfitter and tc p and not tc c,not a collaboration
clas,articles,inspirehep.net,find cn clas and tc p and not tc c,
cms,articles,cds.cern.ch,CMS Papers,
codalema,articles,inspirehep.net,find cn codalema and tc p and not tc c,
cta,articles,inspirehep.net,find cn cta and tc p and not tc c,
d0,articles,inspirehep.net,find cn d0 and tc p and not tc c,
darkside,articles,inspirehep.net,find cn darkside and tc p and not tc c,
delphi,articles,cds.cern.ch,DELPHI Papers,
desi,articles,inspirehep.net,find cn desi and tc p and not tc c,
double chooz,articles,inspirehep.net,find cn double chooz and tc p and not tc c,
dune,articles,inspirehep.net,find cn dune and tc p and not tc c,
eli-np,articles,inspirehep.net,finc cn eli-np and tc p and not tc c,
elisa,articles,inspirehep.net,finc 693__e:elisa and tc p and not tc c,an instrument not a coll.
euclid,articles,inspirehep.net,find cn euclid and tc p and not tc c,
fazia,articles,inspirehep.net,find cn fazia and tc p and not tc c,
fcc,articles,inspirehep.net,find fcc and tc p and not tc c,
fermi-lat,articles,inspirehep.net,find cn fermi-lat and tc p and not tc c,
fopi,articles,inspirehep.net,find cn fopi and tc p and not tc c,
gaspard,articles,inspirehep.net,find cn gaspard and tc p and not tc c,
graal,articles,inspirehep.net,find cn graal and tc p and not tc c,
hades,articles,inspirehep.net,find cn hades and tc p and not tc c,
h1,articles,inspirehep.net,find cn h1 and tc p and not tc c,
hess,articles,inspirehep.net,find (cn hess or cn h e s s) and tc p and not tc c,
ilc,articles,inspirehep.net,finc cn ilc and tc p and not tc c,
integral,articles,inspirehep.net,finc cn integral and tc p and not tc c,
jean-euso,articles,inspirehep.net,find cn jem-euso and tc p and not tc c,
jefferson lab hall-a,articles,inspirehep.net,find cn jefferson lab hall-a and tc p and not tc c,
juno,articles,inspirehep.net,find cn juno and tc p and not tc c,
km3net,articles,inspirehep.net,find cn km3net and tc p and not tc c,
l3,articles,cds.cern.ch,L3 Papers,
lhcb,articles,cds.cern.ch,"LHCb Papers, LHCb Detector Performance Papers",
ligo,articles,inspirehep.net,finc cn ligo and tc p and not tc c,
Lisa-pathfinder,articles,inspirehep.net,finc 693__e:lisa-pathfinder and tc p and not tc c,an instrument not a coll.
lsst,articles,inspirehep.net,find cn lsst and tc p and not tc c,
mami a1,articles,inspirehep.net,find cn mami a1 and tc p and not tc c,
na50,articles,inspirehep.net,find cn na50 and tc p and not tc c,
na60,articles,inspirehep.net,find cn na60 and tc p and not tc c,
nemo-3,articles,inspirehep.net,find cn nemo-3 and tc p and not tc c,
opal,articles,cds.cern.ch,OPAL Papers,
orca,articles,inspirehep.net,find orca and tc p and not tc c,no cn since orca is not an int. coll.
panda,articles,inspirehep.net,find cn panda and tc p and not tc c,
phenix,articles,inspirehep.net,find cn phenix and tc p and not tc c,
planck,articles,inspirehep.net,find cn planck and tc p and not tc c,
snls,articles,inspirehep.net,find cn snls and tc p and not tc c,
solid,articles,inspirehep.net,find cn solid and tc p and not tc c,warning: find articles for SoLid and SoLID !
sox,articles,inspirehep.net,find sox and tc p and not tc c,no cn since sox is not an int. coll.
supernemo,articles,inspirehep.net,find cn supernemo and tc p and not tc c,
thomx,articles,inspirehep.net,find thomx and tc p and not tc c,no cn since thomx is not int. coll.
virgo,articles,inspirehep.net,find cn virgo and tc p and not tc c,
xenon,articles,inspirehep.net,find cn xenon and tc p and not tc c,
,,,,
a1,proceedings,inspirehep.net,find cn a1 and tc c,
auger,proceedings,inspirehep.net,find cn auger and tc c,auger or pierre auger give the same result
aleph,proceedings,inspirehep.net,find cn aleph and tc c,
alice,proceedings,inspirehep.net,find cn alice and tc c,
ams,proceedings,inspirehep.net,find cn ams and tc c,
antares,proceedings,inspirehep.net,find cn antares and tc c,
athena,proceedings,inspirehep.net,find athena and not cn athena and tc c,warning: détecteur X embarqué ESA not CERN coll.
atlas,proceedings,inspirehep.net,find cn atlas and tc c,
babar,proceedings,inspirehep.net,find cn babar and tc c,
belle,proceedings,inspirehep.net,find cn belle and tc c,
belle-ii,proceedings,inspirehep.net,find cn belle-ii and tc c,
borexino,proceedings,inspirehep.net,find cn borexino and tc c,
boss,proceedings,inspirehep.net,find cn boss and tc c,
calice,proceedings,inspirehep.net,find cn calice and tc c,
celeste,proceedings,inspirehep.net,find cn celeste and tc c,
ckmfitter,proceedings,inspirehep.net,find ckmfitter and tc c,not a collaboration
clas,proceedings,inspirehep.net,find cn clas and tc c,
cms,proceedings,inspirehep.net,find cn cms and tc c,
codalema,proceedings,inspirehep.net,find cn codalema and tc c,
cta,proceedings,inspirehep.net,find cn cta and tc c,
d0,proceedings,inspirehep.net,find cn d0 and tc c,
darkside,proceedings,inspirehep.net,find darkside and tc c,no cn since proceeding are also signed by people
delphi,proceedings,cds.cern.ch,DELPHI Conference Proceedings,
desi,proceedings,inspirehep.net,find cn desi and tc c,
double chooz,proceedings,inspirehep.net,find cn double chooz and tc c,
dune,proceedings,inspirehep.net,find cn dune and tc c,
eli-np,proceedings,inspirehep.net,finc cn eli-np and tc c,
elisa,proceedings,inspirehep.net,finc 693__e:elisa and tc c,an instrument not a coll.
euclid,proceedings,inspirehep.net,find cn euclid and tc c,
fazia,proceedings,inspirehep.net,find cn fazia and tc c,
fcc,proceedings,inspirehep.net,find fcc and tc c,
fermi-lat,proceedings,inspirehep.net,find cn fermi-lat and tc c,
fopi,proceedings,inspirehep.net,find cn fopi and tc c,
gaspard,proceedings,inspirehep.net,find cn gaspard and tc c,
graal,proceedings,inspirehep.net,find cn graal and tc c,
hades,proceedings,inspirehep.net,find cn hades and tc c,
h1,proceedings,inspirehep.net,find cn h1 and tc c,
hess,proceedings,inspirehep.net,find (cn hess or cn h e s s) and tc c,
ilc,proceedings,inspirehep.net,finc cn ilc and tc c,
integral,proceedings,inspirehep.net,finc cn integral and tc c,
Jean-euso,proceedings,inspirehep.net,find cn jem-euso and tc c,
jefferson lab hall-a,proceedings,inspirehep.net,find cn jefferson lab hall-a and tc c,
juno,proceedings,inspirehep.net,find cn juno and tc c,
km3net,proceedings,inspirehep.net,find cn km3net and tc c,
l3,proceedings,cds.cern.ch,L3 Conference Proceedings,
lhcb,proceedings,cds.cern.ch,LHCb Conference Proceedings,
ligo,proceedings,inspirehep.net,finc cn ligo and tc c,
Lisa-pathfinder,proceedings,inspirehep.net,finc 693__e:lisa-pathfinder and tc c,
lsst,proceedings,inspirehep.net,find cn lsst and tc c,
mami a1,proceedings,inspirehep.net,find cn mami a1 and tc c,
na50,proceedings,inspirehep.net,find cn na50 and tc c,
na60,proceedings,inspirehep.net,find cn na60 and tc c,
nemo-3,proceedings,inspirehep.net,find cn nemo-3 and tc c,
opal,proceedings,cds.cern.ch,OPAL Conference Proceedings,
orca,proceedings,inspirehep.net,find orca and tc c,no cn since orca is not an int. coll.
panda,proceedings,inspirehep.net,find cn panda and tc c,
phenix,proceedings,inspirehep.net,find cn phenix and tc c,
planck,proceedings,inspirehep.net,find cn planck and tc c,
snls,proceedings,inspirehep.net,find cn snls and tc c,
solid,proceedings,inspirehep.net,find cn solid and tc c,warning: find articles for SoLid and SoLID !
sox,proceedings,inspirehep.net,find sox and tc c,no cn since sox is not an int. coll.
supernemo,proceedings,inspirehep.net,find cn supernemo and tc c,
thomx,proceedings,inspirehep.net,find thomx and tc c,no cn since thomx is not int. coll.
virgo,proceedings,inspirehep.net,find cn virgo and tc c,
xenon,proceedings,inspirehep.net,find cn xenon and tc c,
,,,,
antares,talks,cds.cern.ch,ANTARES Talks,
atlas,talks,cds.cern.ch,ATLAS Conference Slides,
lhcb,talks,cds.cern.ch,LHCb Talks,
,,,,
aleph,theses,cds.cern.ch,ALEPH Theses,
alice,theses,cds.cern.ch,ALICE Theses,
atlas,theses,cds.cern.ch,ATLAS Theses,
cms,theses,cds.cern.ch,CMS Theses,
delphi,theses,cds.cern.ch,DELPHI Theses,
l3,theses,cds.cern.ch,L3 Theses,
lhcb,theses,cds.cern.ch,LHCb Theses,
opal,theses,cds.cern.ch,OPAL Theses,
,,,,
alice,notes,cds.cern.ch,ALICE Public Notes,
atlas,notes,cds.cern.ch,"ATLAS Notes, ATLAS Conference Notes, ATLAS Scientific Notes",
cms,notes,cds.cern.ch,"CMS Notes, CMS Physics Analysis Summaries, CMS Detector Performance Summaries",
delphi,notes,cds.cern.ch,DELPHI Notes,
l3,notes,cds.cern.ch,L3 Notes,
lhcb,notes,cds.cern.ch,"LHCb Notes, LHCb Conference Contributions",
opal,notes,cds.cern.ch,OPAL Notes,
,,,,
alice,reports,cds.cern.ch,ALICE Reports,
atlas,reports,cds.cern.ch,ATLAS Reports,
cms,reports,cds.cern.ch,CMS Reports,
l3,reports,cds.cern.ch,L3 Reports,
lhcb,reports,cds.cern.ch,LHCb Reports,
opal,reports,cds.cern.ch,OPAL Reports,
# -*- coding: utf-8 -*-
""" NAME
deploy my_institute.csv
SYNOPSIS
parametrise the database for a given institute
DESCRIPTION
The parameters for the institute are defined in the CSV file.
It is located scripts/data/my_institute.csv
The definition of the main harvesters are located in
scripts/data/harvesters.csv
The CSV file defining the institute contains the following columns
- domain
- team
- project
- project definition
- comments
The project definition column is split in four:
- collaboration
- instrument, JLAB experiment, etc.
- physicist
- HAL hidden
The CSV file defining the harvesters contains five columns:
- collaboration (identifier)
- controller
- host
- collection
- comment
Firstly, the script create teams, projects and their relations.
Secondly, it create all the harvesters for each project using
OPTIONS