Refined and documented error handling for functions

Author: spookylukey

fluent.runtime/CHANGELOG.rst

@@ -6,6 +6,8 @@ fluent.runtime development version (unreleased)
 
 * Support for Fluent spec 0.8 (``fluent.syntax`` 0.10), including parameterized
   terms.
+* Refined error handling regarding function calls to be more tolerant of errors
+  in FTL files, while silencing developer errors less.
 
 fluent.runtime 0.1 (January 21, 2019)
 -------------------------------------

fluent.runtime/docs/errors.rst

@@ -0,0 +1,43 @@
+Error handling
+==============
+
+The Fluent philosophy is to try to recover from errors, and not throw
+exceptions, on the basis that a partial translation is usually better than one
+that is entirely missing or a 500 page.
+
+python-fluent adopts that philosophy, but also tries to abide by the Zen of
+Python - “Errors should never pass silently. Unless explicitly silenced.”
+
+The combination of these two different philosophies works as follows:
+
+* Errors made by **translators** in the contents of FTL files do not raise
+  exceptions. Instead the errors are collected in the ``errors`` argument returned
+  by ``FluentBundle.format``, and some kind of substitute string is returned.
+  For example, if a non-existent term ``-brand-name`` is referenced from a
+  message, the string ``-brand-name`` is inserted into the returned string.
+
+  Also, if the translator uses a function and passes the wrong number of
+  positional arguments, or unavailable keyword arguments, this error will be
+  caught and reported, without allowing the exception to propagate.
+
+* Exceptions triggered by **developer** errors (whether the authors of
+  python-fluent or a user of python-fluent) are not caught, but are allowed to
+  propagate. For example:
+
+  * An incorrect message ID passed to ``FluentBundle.format`` is most likely a
+    developer error (a typo in the message ID), and so causes an exception to be
+    raised.
+
+    A message ID that is correct but missing in some languages will cause the
+    same error, but it is expected that to cover this eventuality
+    ``FluentBundle.format`` will be wrapped with functions that automatically
+    perform fallback to languages that have all messages defined. This fallback
+    mechanism is outside the scope of ``fluent.runtime`` itself.
+
+  * Message arguments of unexpected types will raise exceptions, since it is the
+    developer's job to ensure the right arguments are being passed to the
+    ``FluentBundle.format`` method.
+
+  * Exceptions raised by custom functions are also assumed to be developer
+    errors (as documented in :doc:`functions`, these functions should not raise
+    exceptions), and are not caught.

fluent.runtime/docs/functions.rst

@@ -0,0 +1,80 @@
+Custom functions
+----------------
+
+You can add functions to the ones available to FTL authors by passing a
+``functions`` dictionary to the ``FluentBundle`` constructor:
+
+.. code-block:: python
+
+    >>> import platform
+    >>> def os_name():
+    ...    """Returns linux/mac/windows/other"""
+    ...    return {'Linux': 'linux',
+    ...            'Darwin': 'mac',
+    ...            'Windows': 'windows'}.get(platform.system(), 'other')
+
+    >>> bundle = FluentBundle(['en-US'], functions={'OS': os_name})
+    >>> bundle.add_messages("""
+    ... welcome = { OS() ->
+    ...    [linux]    Welcome to Linux
+    ...    [mac]      Welcome to Mac
+    ...    [windows]  Welcome to Windows
+    ...   *[other]    Welcome
+    ...   }
+    ... """)
+    >>> print(bundle.format('welcome')[0]
+    Welcome to Linux
+
+These functions can accept positional and keyword arguments, like the ``NUMBER``
+and ``DATETIME`` builtins. They must accept the following types of objects
+passed as arguments:
+
+- unicode strings (i.e. ``unicode`` on Python 2, ``str`` on Python 3)
+- ``fluent.runtime.types.FluentType`` subclasses, namely:
+
+  - ``FluentNumber`` - ``int``, ``float`` or ``Decimal`` objects passed in
+    externally, or expressed as literals, are wrapped in these. Note that these
+    objects also subclass builtin ``int``, ``float`` or ``Decimal``, so can be
+    used as numbers in the normal way.
+  - ``FluentDateType`` - ``date`` or ``datetime`` objects passed in are wrapped in
+    these. Again, these classes also subclass ``date`` or ``datetime``, and can
+    be used as such.
+  - ``FluentNone`` - in error conditions, such as a message referring to an
+    argument that hasn't been passed in, objects of this type are passed in.
+
+Custom functions should not throw errors, but return ``FluentNone`` instances to
+indicate an error or missing data. Otherwise they should return unicode strings,
+or instances of a ``FluentType`` subclass as above. Returned numbers and
+datetimes should be converted to ``FluentNumber`` or ``FluentDateType``
+subclasses using ``fluent.types.fluent_number`` and ``fluent.types.fluent_date``
+respectively.
+
+The type signatures of custom functions are checked before they are used, to
+ensure the right the number of positional arguments are used, and only available
+keyword arguments are used - otherwise a ``TypeError`` will be appended to the
+``errors`` list. Using ``*args`` or ``**kwargs`` to allow any number of
+positional or keyword arguments is supported, but you should ensure that your
+function actually does allow all positional or keyword arguments.
+
+If you want to override the detected type signature (for example, to limit the
+arguments that can be used in an FTL file, or to provide a proper signature for
+a function that has a signature using ``*args`` and ``**kwargs`` but is more
+restricted in reality), you can add an ``ftl_arg_spec`` attribute to the
+function. The value should be a two-tuple containing 1) an integer specifying
+the number of positional arguments, and 2) a list of allowed keyword arguments.
+For example, for a custom function ``my_func`` the following will stop the
+``restricted`` keyword argument from being used from FTL files, while allowing
+``allowed``, and will require that a single positional argument is passed:
+
+.. code-block:: python
+
+    def my_func(arg1, allowed=None, restricted=None):
+        pass
+
+    my_func.ftl_arg_spec = (1, ['allowed'])
+
+The Fluent spec allows keyword arguments with hyphens (``-``) in them.
+Since these cannot be used in valid Python keyword arguments, they are
+disallowed by ``fluent.runtime`` and will be filtered out and generate
+errors if you specify such a keyword in ``ftl_arg_spec`` or use one in a
+message.

fluent.runtime/docs/index.rst

@@ -15,4 +15,6 @@ significant changes.
 
    installation
    usage
+   functions
+   errors
    history

fluent.runtime/docs/usage.rst

@@ -202,54 +202,6 @@ ways:
        >>> val
        'Now is Jun 17, 2018, 3:15:05 PM'
 
-Custom functions
-~~~~~~~~~~~~~~~~
-
-You can add functions to the ones available to FTL authors by passing a
-``functions`` dictionary to the ``FluentBundle`` constructor:
-
-.. code-block:: python
-
-    >>> import platform
-    >>> def os_name():
-    ...    """Returns linux/mac/windows/other"""
-    ...    return {'Linux': 'linux',
-    ...            'Darwin': 'mac',
-    ...            'Windows': 'windows'}.get(platform.system(), 'other')
-
-    >>> bundle = FluentBundle(['en-US'], functions={'OS': os_name})
-    >>> bundle.add_messages("""
-    ... welcome = { OS() ->
-    ...    [linux]    Welcome to Linux
-    ...    [mac]      Welcome to Mac
-    ...    [windows]  Welcome to Windows
-    ...   *[other]    Welcome
-    ...   }
-    ... """)
-    >>> print(bundle.format('welcome')[0]
-    Welcome to Linux
-
-These functions can accept positional and keyword arguments (like the
-``NUMBER`` and ``DATETIME`` builtins), and in this case must accept the
-following types of arguments:
-
--  unicode strings (i.e. ``unicode`` on Python 2, ``str`` on Python 3)
--  ``fluent.runtime.types.FluentType`` subclasses, namely:
--  ``FluentNumber`` - ``int``, ``float`` or ``Decimal`` objects passed
-   in externally, or expressed as literals, are wrapped in these. Note
-   that these objects also subclass builtin ``int``, ``float`` or
-   ``Decimal``, so can be used as numbers in the normal way.
--  ``FluentDateType`` - ``date`` or ``datetime`` objects passed in are
-   wrapped in these. Again, these classes also subclass ``date`` or
-   ``datetime``, and can be used as such.
--  ``FluentNone`` - in error conditions, such as a message referring to
-   an argument that hasn't been passed in, objects of this type are
-   passed in.
-
-Custom functions should not throw errors, but return ``FluentNone``
-instances to indicate an error or missing data. Otherwise they should
-return unicode strings, or instances of a ``FluentType`` subclass as
-above.
 
 Known limitations and bugs
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -268,3 +220,10 @@ Known limitations and bugs
   <https://github.com/andyearnshaw/Intl.js/blob/master/src/12.datetimeformat.js>`_.
 
 Help with the above would be welcome!
+
+
+Other features and further information
+--------------------------------------
+
+* :doc:`functions`
+* :doc:`errors`

fluent.runtime/fluent/runtime/resolver.py

@@ -13,7 +13,8 @@
 
 from .errors import FluentCyclicReferenceError, FluentFormatError, FluentReferenceError
 from .types import FluentDateType, FluentNone, FluentNumber, fluent_date, fluent_number
-from .utils import numeric_to_native, reference_to_id, unknown_reference_error_obj
+from .utils import (args_match, inspect_function_args, numeric_to_native,
+                    reference_to_id, unknown_reference_error_obj)
 
 try:
     from functools import singledispatch
@@ -110,8 +111,8 @@ def fully_resolve(expr, env):
 
 @singledispatch
 def handle(expr, env):
-    raise NotImplementedError("Cannot handle object of type {0}"
-                              .format(type(expr).__name__))
+    raise TypeError("Cannot handle object {0} of type {1}"
+                    .format(expr, type(expr).__name__))
 
 
 @handle.register(Message)
@@ -245,6 +246,7 @@ def handle_variable_reference(argument, env):
                 FluentReferenceError("Unknown external: {0}".format(name)))
         return FluentNone(name)
 
+    # The code below should be synced with fluent.runtime.runtime.handle_argument
     if isinstance(arg_val,
                   (int, float, Decimal,
                    date, datetime,
@@ -383,11 +385,12 @@ def handle_call_expression(expression, env):
                                                .format(function_name)))
         return FluentNone(function_name + "()")
 
-    try:
-        return function(*args, **kwargs)
-    except Exception as e:
-        env.errors.append(e)
-        return FluentNone(function_name + "()")
+    arg_spec = inspect_function_args(function, function_name, env.errors)
+    match, sanitized_args, sanitized_kwargs, errors = args_match(function_name, args, kwargs, arg_spec)
+    env.errors.extend(errors)
+    if match:
+        return function(*sanitized_args, **sanitized_kwargs)
+    return FluentNone(function_name + "()")
 
 
 @handle.register(FluentNumber)

fluent.runtime/fluent/runtime/types.py

@@ -67,6 +67,8 @@ class NumberFormatOptions(object):
     # rather than using underscores as per PEP8, so that
     # we can stick to Fluent spec more easily.
 
+    # Keyword args available to FTL authors must be synced to fluent_number.ftl_arg_spec below
+
     # See https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/NumberFormat
     style = attr.ib(default=FORMAT_STYLE_DECIMAL,
                     validator=attr.validators.in_(FORMAT_STYLE_OPTIONS))
@@ -81,7 +83,7 @@ class NumberFormatOptions(object):
     maximumSignificantDigits = attr.ib(default=None)
 
 
-class FluentNumber(object):
+class FluentNumber(FluentType):
 
     default_number_format_options = NumberFormatOptions()
 
@@ -228,10 +230,24 @@ def fluent_number(number, **kwargs):
     elif isinstance(number, FluentNone):
         return number
     else:
-        raise TypeError("Can't use fluent_number with object {0} for type {1}"
+        raise TypeError("Can't use fluent_number with object {0} of type {1}"
                         .format(number, type(number)))
 
 
+# Specify arg spec manually, for three reasons:
+# 1. To avoid having to specify kwargs explicitly, which results
+#    in duplication, and in unnecessary work inside FluentNumber
+# 2. To stop 'style' and 'currency' being used inside FTL files
+# 3. To avoid needing inspection to do this work.
+fluent_number.ftl_arg_spec = (1, ['currencyDisplay',
+                                  'useGrouping',
+                                  'minimumIntegerDigits',
+                                  'minimumFractionDigits',
+                                  'maximumFractionDigits',
+                                  'minimumSignificantDigits',
+                                  'maximumSignificantDigits'])
+
+
 _UNGROUPED_PATTERN = parse_pattern("#0")
 
 
@@ -255,6 +271,8 @@ class DateFormatOptions(object):
     timeZone = attr.ib(default=None)
 
     # Other
+    # Keyword args available to FTL authors must be synced to fluent_date.ftl_arg_spec below
+
     hour12 = attr.ib(default=None)
     weekday = attr.ib(default=None)
     era = attr.ib(default=None)
@@ -276,7 +294,7 @@ class DateFormatOptions(object):
 _SUPPORTED_DATETIME_OPTIONS = ['dateStyle', 'timeStyle', 'timeZone']
 
 
-class FluentDateType(object):
+class FluentDateType(FluentType):
     # We need to match signature of `__init__` and `__new__` due to the way
     # some Python implementation (e.g. PyPy) implement some methods.
     # So we leave those alone, and implement another `_init_options`
@@ -361,3 +379,19 @@ def fluent_date(dt, **kwargs):
     else:
         raise TypeError("Can't use fluent_date with object {0} of type {1}"
                         .format(dt, type(dt)))
+
+
+fluent_date.ftl_arg_spec = (1,
+                            ['hour12',
+                             'weekday',
+                             'era',
+                             'year',
+                             'month',
+                             'day',
+                             'hour',
+                             'minute',
+                             'second',
+                             'timeZoneName',
+                             'dateStyle',
+                             'timeStyle',
+                             ])