Update BaseReport in order to express conditions as a web2py query.

docs/user/appendix_smartquery.rst

-.. include:: hyperlinks.txt
-
-.. _Appendix E:
-
-Appendix E: smart query
-=======================
-
-The following operators are understood:
-
-* ``&``, ``and``
-* ``|``, ``or``
-* ``~``
-* ``==``, ``=``, ``equal to``, ``equal``, ``equals``, ``is``
-* ``<``, ``less than``
-* ``>``, ``greater than``
-* ``<=``, ``=<``, ``less or equal than``, ``equal or less than``,
-  ``less or equal``, ``equal or less``
-* ``>=``, ``=>``, ``greater or equal than``, ``equal or greater than``,
-  ``greater or equal``, ``equal or greater``
-* ``<>``, ``not equal to``, ``not equal``
-* ``starts with``
-* ``ends with``
-* ``contains``
-* ``in``
-* ``not in``
-

docs/user/conditions.txt

@@ -2,20 +2,27 @@
 
 The fitler conditions is defined in the ``General`` tab.
 
-The ``Conditions`` are used to select a sub-sample of the history records. It
-is based on the smart_query_ available in the Web2py_ framework. The query
-is written in the natural language, but it is limited to field of the
-history table.
+The ``Conditions`` are used to select a sub-sample of the history records.
+It is expressed as a database ``query`` using the syntax of the
+Web2py_ framework.
 
-    .. warning::
+Conditions can be applied to the field of the ``history`` table as well as
+those of its foreign tables.
 
-        Do not use foreign fields, virtual fields, keys of the
-        ``history.data`` dictionary as well as the pseudo field year.
-
-Example of smart query:
+Example of query:
 
     .. code::
 
-        history.id_events = 7 and id_people_categories not in 1,11,13,14
+        (db.events.event == 'People') & (db.history.data.like('%cdd%'))
+
+
+    .. note::
+
+        The ``history.data`` is a dictionary encoded as a JSON string.
+        Therefore, you must use the operator ``like``, ``contains``,
+        ``startswith``, *etc*
 
-Possible operators are given :ref:`Appendix E`.
+        In the JSON_ format, the (key, value) pair is encoded as
+        ``"key": value`` where the value can be string, integer,
+        boolean, *etc*. String are surrounded by ``"`` while boolean
+        value are equal to ``true`` or ``false``.

docs/user/index.rst

@@ -22,7 +22,6 @@ Documentation contents
    appendix_virtualfield
    appendix_column
    appendix_graph
-   appendix_smartquery
 
 Indices and tables
 ==================

languages/fr-fr.py

@@ -74,6 +74,7 @@
 'Data model': 'Modèle des données',
 'Data people': 'Les données des personnes',
 'Database field encoded as tablename.fieldname.': 'Champ de la base de donnée encodé tablename.fieldname.',
+"Database query to select history records:<br> (db.events.event == 'People') & (db.history.data.like('%cdd%'))<br>": "requête pour sélectionner des enregistrements dans la table history:<br> (db.events.event == 'People') & (db.history.data.like('%cdd%'))<br>",
 'database schema': 'schéma de la base de données',
 'Database schema': 'Schéma de la base de données',
 'Define a crude model for the data block associated to each event. The model contains a list of properties. A type is defined to each property or a default value. Valid types are boolean, date, number and string.': 'Defini le modèle du bloc de données associé à chaque évènement. Le modèle contiens une liste de propriétés. Un type est défini pour chaque propriété ou une valeur par défault. Les types sont boolean, date, number et string.',
@@ -282,9 +283,9 @@
 'Period Start': 'Period Start',
 'Person': 'Persone',
 'Persone': 'Persone',
-'PHD': 'PHD',
 'PhD': 'PhD',
 'Phd': 'Phd',
+'PHD': 'PHD',
 'Phd Date': 'Phd Date',
 'Phd defense date': 'Date de la soutenance de la thèse',
 'please input your password again': 'please input your password again',

models/db0_lists.py

@@ -6,11 +6,8 @@ tp_columns = \
 T("Configure the column of the grid displayed in the view.")
 
 tp_conditions = \
-T("Select the history records. "
-  "It can used any field of the history table using the SQL WHERE syntax, "
-  "but the foreign keys are not resolved nor the individual property of "
-  "the history.data dictionary. "
-  "(more information in the smart_query in the web2py documentation).")
+T("Database query to select history records:<br> "
+  "(db.events.event == 'People') & (db.history.data.like('%cdd%'))<br>")
 
 tp_fields = \
 T("Row are grouped according to the value of that field. "

modules/report_objects.py

@@ -20,6 +20,7 @@ MSG_NO_DATAINDEX = "The property dataIndex is required when eval is used."
 MSG_NO_XTYPE = "The property xtype is missing."
 
 REG_DBFIELD = re.compile("\w+\.\w+(?:\.\w+)?", re.UNICODE)
+REG_PYQUERY = re.compile("[\( ]*\w+\.\w+\.\w+")
 REG_SINGLE_DBFIELD = re.compile("^ *\w+\.\w+(\.\w+)? *$", re.UNICODE)
 
 
@@ -164,8 +165,24 @@ class BaseReport(object):
         self.selector = selector
 
         # apply the condition criteria used to filter the history records
+        # condition can be written as a smart query: history.id_events == 7
+        # or like a python query: db.events.event == "People"
         if "conditions" in config:
-            q_conditions = smart_query(db.history, config.conditions)
+            condition = config.conditions
+
+            # minimal protection to avoid injection flow
+            # the beginning of the python query should be like:
+            #    db.table.field
+            #    (db.table.field
+            #    ((db.table.field
+            #    ( ( db.table.field
+            #
+            if REG_PYQUERY.match(condition):
+                q_conditions = eval(condition, None, {"db": db})
+
+            else:
+                q_conditions = smart_query(db.history, config.conditions)
+
             selector.append_query(q_conditions)
 
     def _do_data(self, maps):