Skip to content

GitLab

Commit 66f10ebd authored by LE GAC Renaud's avatar LE GAC Renaud
Browse files

Merge branch '16-plugin-event' into 'master' 

Resolve "Create the plugin_event"

* For `list` and `metric`, filtering conditions can be written using the web2py syntax for database query. Therefore, it can deal with foreign tables
    of the `history` table as well as with the `history.data field`. 
* The `smart_query` is still available but deprecated since it can not handle foreign fields.
* Factorize the application around the `plugin_event`.
* Closes #16 and #19

See merge request !24
parents dec59bd1 c7970e2d
Hide whitespace changes
Inline Side-by-side
Showing with 337 additions and 426 deletions
.gitignore
View file @ 66f10ebd
......@@ -21,5 +21,8 @@ private/
sessions/
static/*-debug.js
static/*-min.js
static/docs/
static/docs/*/
static/plugin_event/docs/*/
static/plugin_event/*-debug.js
static/plugin_event/*-min.js
uploads/
CHANGELOG deleted 100644 → 0
View file @ dec59bd1
--------------------------------- CHANGELOG ----------------------------------
HEAD
- Require plugin_dbui 0.9.7 or higher
- The selector for report uses the accordion layout.
- The form for event uses the card layout.
- In the event table, data model information are displayed using
the Table widget.
- for list and metric, the conditions can be written using the web2py
syntax for database query. Therefore, it can deal with foreign tables
of the history table as well as with the history.data field.
0.6.2 (Nov 2016)
- Migrate to ExtJS 6.
- Require plugin_dbui 0.9.5 and plugin_extjs 6.2.0 or higher.
- New syntax for grid column configuration property:
renderer and summaryRenderer.
- Redesing the JavaScript class UserDataMultiDefault.
- Add the custom widget TrackEvents.grid.HistoryGridFilter.
0.6.0 (Oct 2016)
- Require plugin_dbui 0.9.0 or higher.
- Change the javaScript namespaces to Dbui and TrackEVents.
- Update the database schema by adding the table domains.
- Add the modules matplotlib_tools
- Add the script lbfr_report.py
0.5.6 (Jun 2016)
- Migrate to w2pext 2.14.6 and plugin_dbui 0.8.3
- Release requires web2py 2.14 or higher due to a change in pyDAL.
- New project organization where build_version is in w2pext/uitlities.
- Few bugs fixed.
0.5.3 (Feb 2016)
- Migrate to plugin_dbui 0.8.2
- Apply pslint and jshint rules to the source codes.
- Migrate the API documentation from epydoc to sphinx.
- Update to run via a docker image.
0.5.2 (Jan 2016)
- Use plugin_dbui 0.8.1 and the compress version of the plugins but ace.
- List entries can be sorted in descending order.
- Add a selection criteria on the field history.data in the report selector.
- Fix bugs selecting people categories and quality in reports
- Fix few bugs in the List class.
0.5.0 - 0.5.1 (May 2015)
- Major relealse non backward compatible
- Add the tables objects and object_categories for a more generic tool.
- List, Metric1D and 2D works with missing data. It appends when several
events are merged with different user data blocks.
- Re-enforce column configuration rules: xtype has always to be defined,
dataIndex is required with eval.
- Use the DBURIS construct.
- Add the script fix-duplicate.py to remove duplicate entries in the
history table.
0.4.5 (Mar 2015)
- Major release non-backward compatible.
- Migrate to plugin_dbui 0.6.2.6, pluggin_ace 1.1.8
and to python-pandas 0.15.2
- Migrate from sqlite to mariadb.
- Migrate to matplotlib 1.3.1 instead of Ext.chart.
- Add the user data block in the history / events tables:
MyApp.UserData, MyApp.UserDataMultiDefault,
MyApp.form.plugin.UserModelConsistency and the controller mytools.
- Remove obsolete tables and codes.
- Redesing the viewport using a more generic organisation
(configure, metadata, events, lists, metrics and graphs)
- Use the pGridExport plugin in views/report/grid.html
- Redesing the report section using pandas DataFrame and the
new classes List, Metrid1D and Metric2D.
- Refactor the lists and metrics table and add the graphs table.
- Add user documentation explaining how to create list, metric and
graph. It relies on the python-sphinx 1.3.9.
70753e8 (Jul 14)
- Migrate to plugin_dbui 0.6.1.6 and Web2py 2.9.5.
- Define the length of each string field to garanty Web2py compatibility.
- Use the controllers plugin_dbui/about, documentations and versions.
- Implement new reporting tools based on the javascrip inline script
technique, the Ext.grid.Panel widget and on the views database table.
- Remove the reporting tools based on the plugin_report which is much
more complicated with respect to the new approach.
05d5393 (Mar 14)
- Migrate to plugin_dbui 0.6.0.18 and ExtJS 4.2.1
- This version is not backward compatible
0.3.1-14-gf93ae68 (Sept 13)
- Migrate to plugin_dbui 0.4.15.2 and Ext JS 3.4.1.1
- Incubator for the generic plugin report.
- Report are customized by users via the plugin report.
0.3.0 (Aug 13)
- Rename field stage_xxx as trainee_xxx
- Minor polishing of the interface.
- Add the logic for user loggin.
- Add the help page documentation and about.
0.2.0 (Feb 13)
- Migrate to plugin_dbui 0.4.10.1 to develop filter on date.
- Add the class MySelector, Base, Hardware and Person.
- Build the list of history, hardware and people.
- Simplify the database schema introducing the concept of events
removing budgets, hardware, lines and responsibilities tables.
- Add a new node wizard to enter person and training.
- Count the number of people per category for a given time range.
- Polish the interface
- Rename the table categories, agencies, levels as people_categories,
fundings and organization_level.
- Reports can be extract as a csv, tex and pdf files.
0.1.0 (July 12)
- First running version based on dbui 0.4.7.6
CHANGELOG 0 → 120000
View file @ 66f10ebd
static/plugin_event/CHANGELOG
\ No newline at end of file
LICENSE 0 → 120000
View file @ 66f10ebd
static/plugin_event/LICENSE
\ No newline at end of file
VERSION deleted 100644 → 0
View file @ dec59bd1
0.6.2
\ No newline at end of file
VERSION 0 → 120000
View file @ 66f10ebd
static/plugin_event/VERSION
\ No newline at end of file
configure_build.py 0 → 100644
View file @ 66f10ebd
# -*- coding: utf-8 -*-
"""Configure the build process for the plugin_event
"""
# ............................................................................
#
# JAVASCRIPT LIBRARY
# JavaScript source code and compile version
# paths are relative to the static directory
#
SENCHA_CLS = [
"plugin_event/src",
"plugin_dbui/src",
"plugin_extjs/packages/core/src",
"plugin_extjs/classic/classic/src",
"plugin_extjs/classic/classic/overrides"]
SENCHA_NS = "Ext,Dbui"
JSLIBDEBUG = "plugin_event/event-debug.js"
JSLIBMIN = "plugin_event/event-min.js"
# ...........................................................................
#
# DOCUMENTATION
# output directories for documentation
# paths are relative to the static directory
#
DOCS_DIR = "plugin_event/docs"
JSDOC = "plugin_event/docs/jsduck"
LATEXDOC = "plugin_event/docs/latex"
PDFDOC = "plugin_event/docs/pdf"
# ...........................................................................
#
# PLUGINS
# event
# paths are relative to the application directory
#
PLUGINS_FILES = {}
PLUGINS_FILES["default"] = [
"controllers/plugin_%s.py",
"models/plugin_%s.py",
"modules/plugin_%s",
"private/plugin_%s",
"static/plugin_%s",
"views/plugin_%s"]
PLUGINS_FILES["event"] = [
"controllers/plugin_event.py",
"modules/plugin_event",
"views/plugin_event",
"static/plugin_event/resources/css/event.css",
"static/plugin_event/src",
"static/plugin_event/locale/event-lang-en.js",
"static/plugin_event/locale/event-lang-fr.js",
"static/plugin_event/event-min.js",
"static/plugin_event/docs"]
\ No newline at end of file
controllers/default.py
View file @ 66f10ebd
""" Controllers """
def documentations():
"""Main controller to give access to the documentation.
"""
return dict()
# -*- coding: utf-8 -*-
""" Controllers
"""
@auth.requires(True, requires_login=not request.is_local)
def index():
"""Main Controller to run the application launching the plugin dbui.
......@@ -26,13 +21,7 @@ def index():
else:
session.role = ADMIN
if 'debug' in request.vars:
del request.vars['debug']
url = URL('plugin_dbui', 'debug', args=request.args, vars=request.vars)
else:
url = URL('plugin_dbui', 'index', args=request.args, vars=request.vars)
url = URL('plugin_dbui', 'index', args=request.args, vars=request.vars)
return redirect(url)
......
controllers/mytools.py deleted 100644 → 0
View file @ dec59bd1
""" mytool controllers
A collection of tools specific to the applications
"""
import json
from datetime import datetime
from plugin_dbui import get_id
MSG = T(" - %s entry(ies) modified in the history table. <br>"
" - DO NOT FORGET TO MODIFIED THE REPORT CONFIGURATION.")
MSG_CVT = T(" - Conversion error: %s.")
def userModelConsistency():
""" A user data block is associated to each event.
It is configurable by the user and defined in the events table.
The definition of the user data block (model) and the content
of the history.data field have to be kept in synchronization.
This job is done by this controller.
"""
# decode the request
changes = json.loads(request.vars.changes)
event = request.vars.fieldEvent
model = json.loads(request.vars.fieldData)
# build a map of the modified keys.
# each operation is a list containing (action, old key, new key).
# the action is equal to: add, destroy, update
#
# NOTE: add and delete action are handle via dictionary copy.
#
#
update_keys = {}
for action, oldkey, newkey in changes:
if action != 'update':
continue
# simplify the case where key1 > key2 > key3 to key1 > key3
# remove the case where key1 > key2 > ... > key1
if update_keys:
ok = False
for k, v in update_keys.iteritems():
if oldkey == v:
update_keys[k] = newkey
ok = True
if k == newkey:
del update_keys[k]
break
if ok:
continue
# basic case
update_keys[oldkey] = newkey
# get the event id
id_event = get_id(db.events, event=event)
if id_event == None:
return
# set-up the default values dealing with data conversion
for key in model:
value = model[key]['value']
type = model[key]['type']
if value:
try:
if type == "boolean":
value = value.lower() in ('true', 'vrai', 'y', 'yes', '1')
elif type == "date":
value = datetime.strptime(value, "%Y-%m-%d")
value = value.isoformat()
elif type == "float":
value = float(value)
elif type == "integer":
value = int(value)
if type == "string" and value[0] == "[" and value[-1] == "]":
# reference to a set, pick the first value as default
li = json.loads(value)
value = (li[0] if li else None)
elif type == "reference":
value = None
except ValueError as e:
return MSG_CVT % e
model[key]['value'] = value
# scan and modify the history table
# the delete of old key is implicit since they are not copied
query = db.history.id_events == id_event
rows = db(query).select()
for row in rows:
data = {}
rd = (row.data if isinstance(row.data, dict) else dict())
# copy existing key or add new key
for key in model:
data[key] = (rd[key] if key in rd else model[key]['value'])
# modify key name
for oldkey, newkey in update_keys.iteritems():
if oldkey in rd:
data[newkey] = rd[oldkey]
db(db.history.id == row.id).update(data=data)
return MSG % len(rows)
\ No newline at end of file
controllers/report.py → controllers/plugin_event.py
View file @ 66f10ebd
""" Report controllers
# -*- coding: utf-8 -*-
"""plugin_event controllers
"""
import urllib
......@@ -6,14 +7,21 @@ import json
import re
from plugin_dbui import INLINE_ALERT
from report_objects import (do_title,
Graph,
List,
Metric1D,
Metric2D,
ReportException)
from selector import MySelector, SelectorActiveItemsException
from datetime import datetime
from plugin_dbui import get_id, INLINE_ALERT
from plugin_event import (do_title,
Graph,
List,
Metric1D,
Metric2D,
ReportException)
from plugin_event import MySelector, SelectorActiveItemsException
MSG = T(" - %s entry(ies) modified in the history table. <br>"
" - DO NOT FORGET TO MODIFIED THE REPORT CONFIGURATION.")
MSG_CVT = T(" - Conversion error: %s.")
def graph_extjs():
......@@ -126,3 +134,111 @@ def grid():
# delegate the grid rendering to the view
return dict(cfg_store=store, grid=grid, title=title)
def userModelConsistency():
""" A user data block is associated to each event.
It is configurable by the user and defined in the events table.
The definition of the user data block (model) and the content
of the history.data field have to be kept in synchronization.
This job is done by this controller.
"""
# decode the request
changes = json.loads(request.vars.changes)
event = request.vars.fieldEvent
model = json.loads(request.vars.fieldData)
# build a map of the modified keys.
# each operation is a list containing (action, old key, new key).
# the action is equal to: add, destroy, update
#
# NOTE: add and delete action are handle via dictionary copy.
#
#
update_keys = {}
for action, oldkey, newkey in changes:
if action != 'update':
continue
# simplify the case where key1 > key2 > key3 to key1 > key3
# remove the case where key1 > key2 > ... > key1
if update_keys:
ok = False
for k, v in update_keys.iteritems():
if oldkey == v:
update_keys[k] = newkey
ok = True
if k == newkey:
del update_keys[k]
break
if ok:
continue
# basic case
update_keys[oldkey] = newkey
# get the event id
id_event = get_id(db.events, event=event)
if id_event == None:
return
# set-up the default values dealing with data conversion
for key in model:
value = model[key]['value']
type = model[key]['type']
if value:
try:
if type == "boolean":
value = value.lower() in ('true', 'vrai', 'y', 'yes', '1')
elif type == "date":
value = datetime.strptime(value, "%Y-%m-%d")
value = value.isoformat()
elif type == "float":
value = float(value)
elif type == "integer":
value = int(value)
if type == "string" and value[0] == "[" and value[-1] == "]":
# reference to a set, pick the first value as default
li = json.loads(value)
value = (li[0] if li else None)
elif type == "reference":
value = None
except ValueError as e:
return MSG_CVT % e
model[key]['value'] = value
# scan and modify the history table
# the delete of old key is implicit since they are not copied
query = db.history.id_events == id_event
rows = db(query).select()
for row in rows:
data = {}
rd = (row.data if isinstance(row.data, dict) else dict())
# copy existing key or add new key
for key in model:
data[key] = (rd[key] if key in rd else model[key]['value'])
# modify key name
for oldkey, newkey in update_keys.iteritems():
if oldkey in rd:
data[newkey] = rd[oldkey]
db(db.history.id == row.id).update(data=data)
return MSG % len(rows)
docs/api/callback.rst
View file @ 66f10ebd
......@@ -7,9 +7,9 @@ A collections of functions to be call before a database operation is completed.
Functions
^^^^^^^^^
.. currentmodule:: callbacks
.. currentmodule:: plugin_event
.. autosummary::
:toctree: generated/
~INHIBIT_CASCADE_DELETE
INHIBIT_CASCADE_DELETE
docs/api/conf.py
View file @ 66f10ebd
......@@ -78,6 +78,7 @@ source_suffix = '.rst'
master_doc = 'index'
# General information about the project.
app = 'track_events'
now = datetime.now()
project = u'track_events API'
copyright = u'2012-%s, R. Le Gac, licensed under CeCILL' % now.year
......@@ -204,7 +205,7 @@ html_theme = 'sphinxdoc'
# html_file_suffix = None
# Output file base name for HTML help builder.
htmlhelp_basename = 'track_events_api_doc'
htmlhelp_basename = '%s_api_doc' % app
# -- Options for LaTeX output --------------------------------------------------
......@@ -218,14 +219,23 @@ latex_elements = {
# Additional stuff for the LaTeX preamble.
# 'preamble': '',
# Latex figure (float) alignment
'figure_align': 'htb',
# fancy chapter title
'fncychap': '\\usepackage[Glenn]{fncychap}',
# Use the Computer Modern fonts instead of Times and Helvetica
'fontpkg': ''
}
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title, author, documentclass [howto/manual]).
latex_documents = [
(master_doc,
'track_events_api.tex',
u'track\\_events API',
'%s_api.tex' % app,
u'%s API documentation' % app.replace('_', '\\_'),
u'R. Le Gac',
'manual'),
]
......@@ -256,7 +266,7 @@ latex_documents = [
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
(master_doc, 'track_events_api', u'API',
(master_doc, '%s_api' %app, u'%s API' % app,
[author], 1)
]
......@@ -270,8 +280,8 @@ man_pages = [
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
(master_doc, 'track_events_api', u'track_events API',
author, 'track_events_api', 'One line description of project.',
(master_doc, str(app), u'%s API' % app,
author, '%s_api' % app, 'Documentation for the Python API.',
'Miscellaneous'),
]
......
docs/api/generated/basereport/report_objects.BaseReport.to_df.rst deleted 100644 → 0
View file @ dec59bd1
report_objects.BaseReport.to_df
===============================
.. currentmodule:: report_objects
.. automethod:: BaseReport.to_df
\ No newline at end of file
docs/api/generated/callbacks.INHIBIT_CASCADE_DELETE.rst deleted 100644 → 0
View file @ dec59bd1
callbacks.INHIBIT_CASCADE_DELETE
================================
.. currentmodule:: callbacks
.. autofunction:: INHIBIT_CASCADE_DELETE
\ No newline at end of file
docs/api/generated/graph/report_objects.Graph.to_df.rst deleted 100644 → 0
View file @ dec59bd1
report_objects.Graph.to_df
==========================
.. currentmodule:: report_objects
.. automethod:: Graph.to_df
\ No newline at end of file
docs/api/generated/graph/report_objects.Graph.to_pdf.rst deleted 100644 → 0
View file @ dec59bd1
report_objects.Graph.to_pdf
===========================
.. currentmodule:: report_objects
.. automethod:: Graph.to_pdf
\ No newline at end of file
docs/api/generated/graph/report_objects.Graph.to_png.rst deleted 100644 → 0
View file @ dec59bd1
report_objects.Graph.to_png
===========================
.. currentmodule:: report_objects
.. automethod:: Graph.to_png
\ No newline at end of file
docs/api/generated/graph/report_objects.Graph.to_svg.rst deleted 100644 → 0
View file @ dec59bd1
report_objects.Graph.to_svg
===========================
.. currentmodule:: report_objects
.. automethod:: Graph.to_svg
\ No newline at end of file
docs/api/generated/list/report_objects.List.to_df.rst deleted 100644 → 0
View file @ dec59bd1
report_objects.List.to_df
=========================