|\ merge from release-0.6, released as 0.6.

ChangeLog.rst

@@ -1,17 +1,37 @@
-ayrton (0.5) UNRELEASED; urgency=medium
+ayrton (0.6) UNRELEASED; urgency=medium
+
+  * Great improvements in `remote()`'s API and sematics:
+    * Made sure local varaibles go to and come back from the remote.
+    * Code block is executes syncronically.
+    * For the moment the streams are no longer returned.
+    * _python_only option is gone.
+    * Most tests actually connect to a listening netcat, only one test uses `ssh`.
+  * Fixed bugs in the new parser.
+  * Fixed globals/locals mix up.
+  * Scripts are no longer wrapped in a function. This means that you can't return values and that module semantics are restored.
+  * `ayrton` exits with status 1 when the script fails to run (SyntaxError, etc).
+
+ -- Marcos Dione <[email protected]>  Wed, 28 Oct 2015 20:57:19 +0100
+
+ayrton (0.5) unstable; urgency=medium
 
-  * source() is out. use python's import system.
-  * Support executing foo.py().
   * Much better command detection.
-  * CommandNotFound exception is now a subclass of NameError.
-  * Allow Command keywords be named like '-l' and '--long-option', so it supports options with dashes.
+  * `CommandNotFound` exception is now a subclass of `NameError`.
+  * Allow `Command` keywords be named like `-l` and `--long-option`, so it supports options with single dashes (`-long-option`, à la `find`).
   * This also means that long-option is no longer passed as --long-option; you have to put the dashes explicitly.
   * bash() does not return a single string by default; override with single=True.
   * Way more tests.
   * Updated docs.
 
  -- Marcos Dione <[email protected]>  Sun, 30 Aug 2015 15:13:30 +0200
 
+ayrton (0.4.4) unstable; urgency=low
+
+  * `source()` is out. use Python's import system.
+  * Support executing `foo.py()`.
+
+ -- Marcos Dione <[email protected]>  Wed, 20 May 2015 23:44:42 +0200
+
 ayrton (0.4.3) unstable; urgency=medium
 
   * Let commands handle SIGPIE and SIGINT. Python does funky things to them.
@@ -24,7 +44,7 @@ ayrton (0.4.3) unstable; urgency=medium
 ayrton (0.4.2) unstable; urgency=low
 
   * _bg allows running a command in the background.
-  * _fails allows a Command to fail even when option('-e') is on. 
+  * _fails allows a Command to fail even when option('-e') is on.
   * Try program_name as program-name if the first failed the path lookup.
   * Convert all arguments to commands to str().
   * chdir() is an alias of cd().
@@ -41,7 +61,7 @@ ayrton (0.4.2) unstable; urgency=low
 ayrton (0.4) unstable; urgency=low
 
   * >= can redirect stederr to stdout.
-  * o(option=argument) can be used to declare keyword params among/before 
+  * o(option=argument) can be used to declare keyword params among/before
     positional ones.
   * bash() now returns a single string if there is only one result.
   * Slightly better error reporting: don't print a part of the stacktrace

Makefile

@@ -3,7 +3,10 @@ all: docs
 INSTALL_DIR=$(HOME)/local
 
 tests:
-	python3 -m unittest discover -v ayrton
+	bash -c 'while true; do nc -l -s 127.0.0.1 -p 2233 -e /bin/bash; done' & \
+	pid=$$!; \
+	LC_ALL=C python3 -m unittest discover -v ayrton; \
+	kill $$pid
 
 docs:
 	PYTHONPATH=${PWD} make -C doc html

README.md

@@ -14,7 +14,7 @@ Nothing fancy, right? Let's try something slightly different:
 
     echo ('Hello, World!')
 
-Not interested yet? What if I tell you that that `echo` function just
+Not interested yet? What if I tell you that `echo` function just
 executed `/bin/echo`?:
 
     mdione@diablo:~/src/projects/ayrton$ strace -e process -ff ayrton doc/examples/hw.ay
@@ -153,20 +153,15 @@ The cherry on top of the cake, or more like the melon of top of the cupcake, is
 (semi) transparent remote execution. This is achieved with the following construct:
 
     a= 42
-    with remote ('localhost') as streams:
-        foo= input ()
-        print (foo)
+    with remote ('localhost'):
         # we can also access variables already in the scope
         # even when we're actually running in another machine
         print (a)
+        # we can modify those variables
+        a= 27
 
-    # streams returns 3 streams: stdin, stdout, stderr
-    (i, o, e)= streams
-    # notice that we must include the \n at the end so input() finishes
-    # and that you must transmit bytes only, no strings
-    i.write (b'bar\n')
-    print (o.readlines ())
-
+    # and those modifications are reflected locally
+    assert (a, 27)
 
 The body of the `with remote(): ...` statement is actually executed in a remote
 machine after connecting via `ssh`. The `remote()` context manager accepts the

TODO.rst

@@ -14,6 +14,8 @@ Really do:
 
 * becareful with if cat () | grep (); error codes must be carried too
 
+* exit code of last Command should be used as return code of functions
+
 * process substitution
 
    * https://github.com/amoffat/sh/issues/66
@@ -46,4 +48,3 @@ Think deeply about:
     * -f vs (-)f vs _f
 * commands in keywords should also be _out=Capture
 * which is the sanest default, bash (..., single=True) or otherwise
-* foo(-l, --long-option)?

ayrton/__init__.py

@@ -22,70 +22,43 @@
 import importlib
 import ast
 import logging
+import dis
 
-# logging.basicConfig(filename='ayrton.%d.log' % os.getpid (),level=logging.DEBUG)
-# logging.basicConfig(filename='ayrton.log',level=logging.DEBUG)
+# patch logging so we have debug2 and debug3
+import ayrton.utils
+
+log_format= "%(asctime)s %(name)16s:%(lineno)-4d (%(funcName)-21s) %(levelname)-8s %(message)s"
+date_format= "%H:%M:%S"
+
+# uncomment one of these for way too much debugging :)
+# logging.basicConfig(filename='ayrton.%d.log' % os.getpid (), level=logging.DEBUG, format=log_format, datefmt=date_format)
+# logging.basicConfig(filename='ayrton.log', level=logging.DEBUG, format=log_format, datefmt=date_format)
 logger= logging.getLogger ('ayrton')
 
 # things that have to be defined before importing ayton.execute :(
-# singleton
+# singleton needed so the functions can access the runner
 runner= None
 
 from ayrton.castt import CrazyASTTransformer
 from ayrton.execute import o, Command, Capture, CommandFailed
 from ayrton.parser.pyparser.pyparse import CompileInfo, PythonParser
 from ayrton.parser.astcompiler.astbuilder import ast_from_node
+from ayrton.ast_pprinter import pprint
 
-__version__= '0.5'
+__version__= '0.6'
 
 def parse (script, file_name=''):
     parser= PythonParser (None)
     info= CompileInfo (file_name, 'exec')
     return ast_from_node (None, parser.parse_source (script, info), info)
 
-class Ayrton (object):
-    def __init__ (self, globals=None, **kwargs):
-        if globals is None:
-            self.globals= {}
-        else:
-            self.globals= globals
-        polute (self.globals, kwargs)
-
-        self.options= {}
-        self.pending_children= []
-
-    def run_file (self, file):
-        # it's a pity that parse() does not accept a file as input
-        # so we could avoid reading the whole file
-        return self.run_script (open (file).read (), file)
-
-    def run_script (self, script, file_name):
-        tree= parse (script, file_name)
-        tree= CrazyASTTransformer (self.globals, file_name).modify (tree)
-
-        return self.run_tree (tree, file_name)
-
-    def run_tree (self, tree, file_name):
-        logger.debug (ast.dump (tree))
-        return self.run_code (compile (tree, file_name, 'exec'))
-
-    def run_code (self, code):
-        locals= {}
-        exec (code, self.globals, locals)
-
-        return locals['ayrton_return_value']
-
-    def wait_for_pending_children (self):
-        for i in range (len (self.pending_children)):
-            child= self.pending_children.pop (0)
-            child.wait ()
-
 def polute (d, more):
     d.update (__builtins__)
     # weed out some stuff
     for weed in ('copyright', '__doc__', 'help', '__package__', 'credits', 'license', '__name__'):
         del d[weed]
 
+    # envars as gobal vars, shell like
     d.update (os.environ)
 
     # these functions will be loaded from each module and put in the globals
@@ -123,18 +96,148 @@ def polute (d, more):
 
     d.update (more)
 
-def run_tree (tree, globals):
+class Ayrton (object):
+    def __init__ (self, g=None, l=None, **kwargs):
+        logger.debug ('new interpreter')
+        logger.debug3 ('globals: %s', ayrton.utils.dump_dict (g))
+        logger.debug3 ('locals: %s', ayrton.utils.dump_dict (l))
+        if g is None:
+            self.globals= {}
+        else:
+            self.globals= g
+        polute (self.globals, kwargs)
+
+        if l is None:
+            # If exec gets two separate objects as globals and locals,
+            # the code will be executed as if it were embedded in a class definition.
+            # and this happens:
+            """
+            In [7]: source='''import math
+            ...: def foo ():
+            ...:     math.floor (1.1, )
+            ...:
+            ...: foo()'''
+
+            In [8]: import ast
+            In [9]: t= ast.parse (source)
+            In [10]: c= compile (t, 'foo.py', 'exec')
+            In [11]: exec (c, {}, {})
+            ---------------------------------------------------------------------------
+            NameError                                 Traceback (most recent call last)
+            <ipython-input-11-3a2844c232c1> in <module>()
+            ----> 1 exec (c, {}, {})
+            /home/mdione/src/projects/ayrton/foo.py in <module>()
+            /home/mdione/src/projects/ayrton/foo.py in foo()
+            NameError: name 'math' is not defined
+            """
+            self.locals= self.globals
+        else:
+            self.locals= l
+
+        self.options= {}
+        self.pending_children= []
+
+    def run_file (self, file):
+        # it's a pity that parse() does not accept a file as input
+        # so we could avoid reading the whole file
+        return self.run_script (open (file).read (), file)
+
+    def run_script (self, script, file_name):
+        tree= parse (script, file_name)
+        # TODO: self.locals?
+        tree= CrazyASTTransformer (self.globals, file_name).modify (tree)
+
+        return self.run_tree (tree, file_name)
+
+    def run_tree (self, tree, file_name):
+        logger.debug2 ('AST: %s', ast.dump (tree))
+        logger.debug2 ('code: \n%s', pprint (tree))
+
+        return self.run_code (compile (tree, file_name, 'exec'))
+
+    def run_code (self, code):
+        if logger.parent.level<=logging.DEBUG2:
+            logger.debug2 ('------------------')
+            logger.debug2 ('main (gobal) code:')
+            handler= logger.parent.handlers[0]
+
+            handler.acquire ()
+            dis.dis (code, file=handler.stream)
+            handler.release ()
+
+            for inst in dis.Bytecode (code):
+                if inst.opname=='LOAD_CONST':
+                    if type (inst.argval)==type (code):
+                        logger.debug ('------------------')
+                        handler.acquire ()
+                        dis.dis (inst.argval, file=handler.stream)
+                        handler.release ()
+                    elif type (inst.argval)==str:
+                        logger.debug ("last function is called: %s", inst.argval)
+
+        '''
+        exec(): If only globals is provided, it must be a dictionary, which will
+        be used for both the global and the local variables. If globals and locals
+        are given, they are used for the global and local variables, respectively.
+        If provided, locals can be any mapping object. Remember that at module
+        level, globals and locals are the same dictionary. If exec gets two
+        separate objects as globals and locals, the code will be executed as if
+        it were embedded in a class definition.
+
+        If the globals dictionary does not contain a value for the key __builtins__,
+        a reference to the dictionary of the built-in module builtins is inserted
+        under that key. That way you can control what builtins are available to
+        the executed code by inserting your own __builtins__ dictionary into
+        globals before passing it to exec().
+
+        The default locals act as described for function locals() below:
+        modifications to the default locals dictionary should not be attempted.
+        Pass an explicit locals dictionary if you need to see effects of the code
+        on locals after function exec() returns.
+
+        locals(): Update and return a dictionary representing the current local
+        symbol table. Free variables are returned by locals() when it is called
+        in function blocks, but not in class blocks.
+
+        The contents of this dictionary should not be modified; changes may not
+        affect the values of local and free variables used by the interpreter.
+        '''
+        error= None
+        try:
+            exec (code, self.globals, self.locals)
+        except Exception as e:
+            error= e
+
+        logger.debug3 ('globals at script exit: %s', ayrton.utils.dump_dict (self.globals))
+        logger.debug3 ('locals at script exit: %s', ayrton.utils.dump_dict (self.locals))
+        result= self.locals.get ('ayrton_return_value', None)
+        logger.debug ('ayrton_return_value: %r', result)
+
+        if error is not None:
+            raise error
+
+        return result
+
+    def wait_for_pending_children (self):
+        for i in range (len (self.pending_children)):
+            child= self.pending_children.pop (0)
+            child.wait ()
+
+def run_tree (tree, g, l):
+    """main entry point for remote()"""
     global runner
-    runner= Ayrton (globals=globals)
-    runner.run_tree (tree, 'unknown_tree')
+    runner= Ayrton (g=g, l=l)
+    return runner.run_tree (tree, 'unknown_tree')
 
-def run_file_or_script (script=None, file=None, **kwargs):
+def run_file_or_script (script=None, file='script_from_command_line', **kwargs):
+    """Main entry point for bin/ayrton and unittests."""
+    logger.debug ('===========================================================')
     global runner
     runner= Ayrton (**kwargs)
     if script is None:
         v= runner.run_file (file)
     else:
-        v= runner.run_script (script, 'script_from_command_line')
+        v= runner.run_script (script, file)
 
     return v