add lisp packages

lisp/org/testing/README

@@ -0,0 +1,157 @@
+# -*- mode:org -*-
+#+TITLE: Org mode Testing
+#+PROPERTY: header-args:emacs-lisp :results silent
+
+* Dependencies
+
+The only dependency is [[http://www.emacswiki.org/emacs/ErtTestLibrary][ERT]] the Emacs testing library which ships with
+Emacs24.  If you are running an older version of Emacs and don't
+already have ERT installed it can be installed from its old [[https://github.com/ohler/ert][git
+repository]].
+
+* Non-interactive batch testing from the command line
+
+The simplest way to run the Org mode test suite is from the command
+line with the following invocation.  Note that the paths below are
+relative to the base of the Org mode directory.
+
+Also note that many of the current tests uses babel evaluation...
+
+#+BEGIN_SRC sh :dir (expand-file-name "..")
+  # For Emacs earlier than 24, add -L /path/to/ert
+  emacs -Q --batch \
+        -L lisp/ -L testing/ -L testing/lisp -l lisp/org.el \
+        -l lisp/org-id.el -l testing/org-test.el \
+        --eval "(progn (org-reload) (setq org-confirm-babel-evaluate nil) \
+        (org-babel-do-load-languages 'org-babel-load-languages \
+        '((emacs-lisp . t) (shell . t) (org . t))))" \
+        -f org-test-run-batch-tests
+#+END_SRC
+
+The options in the above command are explained below.
+
+| -Q      | ignores any personal configuration ensuring a vanilla Emacs instance is used |
+| --batch | runs Emacs in "batch" mode with no gui and termination after execution       |
+| -l      | loads Org mode and the Org mode test suite defined in testing/org-test.el    |
+| --eval  | reloads Org mode and allows evaluation of code blocks by the tests           |
+| -f      | actually runs the tests using the `org-test-run-batch-tests' function        |
+
+* Trigger the tests with 'make'
+
+** Recompile all
+
+Target ~test~ can be used to trigger a test run.  The tests start
+after cleaning up and recompilation.
+
+#+BEGIN_SRC sh :dir (expand-file-name "..") :results silent
+make test
+#+END_SRC
+
+See ../mk/default.mk for details.
+
+** Test dirty
+
+The 'dirty' targets are for recompiling without cleaning and
+rebuilding everything.  This usually speeds up the recompilation
+considerably.  Note that this speed up comes to the price of possibly
+weird errors due to the unclean build.
+
+The dirty target for testing is called ~test-dirty~.
+
+#+BEGIN_SRC sh :dir (expand-file-name "..") :results silent
+make test-dirty
+#+END_SRC
+
+** Select tests by regexp
+
+Variable ~BTEST_RE~ can be set to limit the tests which are performed.
+~BTEST_RE~ is interpreted as regexp.
+
+Example:
+
+#+begin_src shell
+make BTEST_RE='test-.*-inlinetask' test-dirty
+#+end_src
+
+yields
+
+#+begin_example
+...
+selected tests: test-.*-inlinetask
+Running 2 tests (2017-12-28 15:04:45+0100)
+   passed  1/2  test-org-export/handle-inlinetasks
+   passed  2/2  test-org-inlinetask/goto-end
+
+Ran 2 tests, 2 results as expected (2017-12-28 15:04:45+0100)
+...
+#+end_example
+
+* Interactive testing from within Emacs
+
+To run the Org mode test suite from a current Emacs instance simply
+load and run the test suite with the following commands.
+
+1) First load the test suite.
+   #+BEGIN_SRC emacs-lisp :var here=(buffer-file-name)
+     (add-to-list 'load-path (file-name-directory here))
+     (require 'org-test)
+   #+END_SRC
+
+2) Load required Babel languages
+   #+BEGIN_SRC emacs-lisp
+     (org-babel-do-load-languages
+      'org-babel-load-languages
+      (and
+       (mapc (lambda (lang) (add-to-list 'org-babel-load-languages (cons lang t)))
+             '(emacs-lisp shell org))
+       org-babel-load-languages))
+   #+END_SRC
+
+3) Then run the test suite.  Babel evaluation confirmation is disabled
+   and ~C-c C-c~ is enabled while running the tests.
+   #+BEGIN_SRC emacs-lisp
+     (let (org-babel-no-eval-on-ctrl-c-ctrl-c
+           org-confirm-babel-evaluate)
+       (org-test-run-all-tests))
+   #+END_SRC
+
+   When a test fails, run it interactively and investigate the problem
+   in the ERT results buffer.
+
+   To run one test: Use this as a demo example of a failing test
+   #+BEGIN_SRC emacs-lisp
+     (ert-deftest test-org/org-link-encode-ascii-character-demo-of-fail ()
+       (should (string= "%5B"  ; Expecting %5B is correct.
+                        (org-link-encode "[")))
+       (should (string= "%5C"  ; Expecting %5C is wrong, %5D correct.
+                        (org-link-encode "]"))))
+   #+END_SRC
+   or evaluate the ~ert-deftest form~ of the test you want to run.
+   Then ~M-x ert RET
+   test-org/org-link-encode-ascii-character-demo-of-fail RET~.  When
+   not visible yet switch to the ERT results buffer named ~*ert*~.
+   When a test failed the ERT results buffer shows the details of the
+   first ~should~ that failed.  See ~(info "(ert)Running Tests
+   Interactively")~ on how to re-run, start the debugger etc.
+
+   To run several tests: ~M-x ert RET "<your regexp here>" RET~.
+
+   To run all tests of a single test file: ~M-x ert-delete-all-tests
+   RET~ and confirm.  ~M-x load-file RET testing/lisp/<file>.el RET
+   M-x ert RET t RET~.
+
+   Consider to set
+   #+BEGIN_SRC emacs-lisp
+     (setq pp-escape-newlines nil)
+   #+END_SRC
+   before running the test when looking at ~should~ in the ERT results
+   buffer.  Especially when using ~l~ to look at passed test results
+   and possibly missing an appropriate setting of ~pp-escape-newlines~
+   made only temporarily for the running time of the test as
+   e. g. tests using ~org-test-table-target-expect-tblfm~ do.
+
+* Troubleshooting
+
+- If the variable ~org-babel-no-eval-on-ctrl-c-ctrl-c~ is non-nil then
+  it will result in some test failure, as there are tests which rely
+  on this behavior.