rstdoc¶

Purpose¶

rstdoc(1) Version 1.8.2 | rstdoc¶

Many companies use DOCX and thus produce an information barrier. Working with text is more integrated in the (software) development process. A final format can be DOCX, but, at least during development, text is better.

Sphinx is an extension of Docutils used for many (software) projects, but it does not support creation of DOCX files, which certain companies demand. Pandoc does support DOCX, but does not support the Sphinx extensions, hence :ref: and the like cannot be used.

This python package supports working with RST as documentation format without depending on Sphinx.

link RST documents using substitutions (generated in _links_xxx.r?st)
create a .tags file to jump around in an editor that support ctags
RST handling with python: reformat/create RST tables
post-process Pandoc’s conversion from DOCX to RST
pre-process Pandoc’s conversion from RST to DOCX
Support in building with WAF (or Makefile)
- expand SimpleTemplate template files .stpl
- graphics files (.tikz, .svg, .dot, .uml, .eps or .stpl thereof, and .pyg) are converted to .png and placed into ./_images or <updir>/_images, if there, else into current directory.
- a gen file specifies how RST should be generated from source code files (see dcx.py)

The conventions used are shown

by the example produced via rstdcx --rest/--stpl/--over/--ipdt smplfldr
by the documentation sources that can be found at https://github.com/rstdoc/rstdoc/tree/master/doc

pip install rstdoc installs:

Module	CLI Script	Description
dcx	rstdcx, rstdoc	create `.tags`, labels and links
fromdocx	rstfromdocx	Convert DOCX to RST using Pandoc
listtable	rstlisttable	Convert RST grid tables to list-tables
untable	rstuntable	Converts certain list-tables to paragraphs
reflow	rstreflow	Reflow paragraphs and tables
reimg	rstreimg	Rename images referenced in the RST file
retable	rstretable	Transforms list tables to grid tables

rstdcx¶

restructuredText sources are split into two types of files: main files considered by Sphinx, and included files. Which of .rest or .rst is main or included is determined by source_suffix in a <root>/conf.py or opposite to the extension of the included _links_sphinx.r?st file:

if you have .. include:: /_links_sphinx.rest, then the main file extension is .rst

rstdoc creates documentation (PDF, HTML, DOCX) from restructuredText (.rst, .rest) using either

rstdoc and rstdcx command line tools call dcx.py. which

creates .tags to jump around with the editor
handles .stpl files
processes gen files (see examples produced by –rest)
creates links files (_links_docx.r?st, _links_sphinx.r?st, …)
forwards known files to either Pandoc, Sphinx or Docutils

See example at the end of dcx.py. It is supposed to be used with a build tool. make and waf examples are included.

Initialize example tree (add --rstrest to make .rst main and .rest included files):

$ ./dcx.py --rest repo #repo/doc/{sy,ra,sr,dd,tp}.rest files OR
$ ./dcx.py --stpl repo #repo/doc/{sy,ra,sr,dd,tp}.rest.stpl files
$ ./dcx.py --ipdt repo #repo/pdt/AAA/{i,p,d,t}.rest.stpl files
$ ./dcx.py --over repo #.rest all over

Only create .tags and _links_xxx.r?st:
```
$ cd repo
$ rstdoc
```
Create the docs (and .tags and _links_xxx.r?st) with make:
```
$ make html #OR
$ make epub #OR
$ make latex #OR
$ make docx #OR
$ make pdf
```
The latter two are done by Pandoc, the others by Sphinx.
Create the docs (and .tags and _links_xxx.r?st) with waf:

Instead of using make one can load dcx.py (rstdoc.dcx) in waf. waf also considers all recursively included files, such that a change in any of them results in a rebuild. All files can have an additional .stpl extension to use SimpleTemplate.

$ waf configure #also copies the latest version of waf in here $ waf –docs docx,sphinx_html,rst_odt $ #or you provide –docs during configure to always compile the docs
- rst_xxx: via rst2xxx.py
- sphinx_xxx: via Sphinx and
- just xxx: via Pandoc.

The following image language files should be parallel to the .r?st files. They are automatically converted to .png and placed into ./_images or <updir>/_images or else parallel to the file.

.tikz or .tikz.stpl. This needs LaTex.
.svg or .svg.stpl
.dot or .dot.stpl

This needs graphviz.
.uml or .uml.stpl

This needs plantuml . Provide either
- plantuml.bat with e.g. java -jar "%~dp0plantuml.jar" %* or
- plantuml sh script with java -jar `dirname $BASH_SOURCE`/plantuml.jar "$@"
.eps or .eps.stpl embedded postscript files.

This needs inkscape.
.pyg contains python code that produces a graphic. If the python code defines a to_svg or a save_to_png function, then that is used, to create a png. Else the following is tried
- pyx.canvas.canvas from the pyx library or
- cairocffi.Surface from cairocffi
- matplotlib. If matplotlib.pyplot.get_fignums()>1 the figures result in <name><fignum>.png
The same code or the file names can be used in a .r?st.stpl file with pngembed() or dcx.svgembed() to embed in html output.
```
{{!svgembed("egpyx.pyg",outinfo)}}
<%
ansvg=svgembed('''
from svgwrite import cm, mm, drawing
d=drawing.Drawing(viewBox=('0 0 300 300'))
d.add(d.circle(center=(2*cm, 2*cm), r='1cm', stroke='blue', stroke_width=9))
'''.splitlines(),outinfo)
%>
{{!ansvg}}
```

Conventions¶

Files

main files and included files: .rest, .rst or vice versa. .txt are for literally included files (use :literal: option).

templates separately rendered : *.rest.stpl and *.rst.stpl template included: *.rst.tpl Template lookup is done in . and .. with respect to the current file.

with %include('some.rst.tpl', param="test") with optional parameters

with %globals().update(include('utility.rst.tpl')) if it contains only definitions

Links

.. _`id`: are reST targets. reST targets should not be template-generated. The template files should have a higher or equal number of targets than the generated file, in order for tags to jump to the template original. If one wants to generate reST targets, then this should better happen in a previous step, e.g. with gen files mentioned above.
References use replacement substitutions: |id|.
If you want an overview of the linking (traceability), add .. include:: _traceability_file.rst to index.rest or another .rest parallel to it. It is there in the example project, to include it in tests. _traceability_file.{svg,png,rst} are all in the same directory.

Link files are created in link roots, which are folders where the first main file (.rest or .rst) is encoutered during depth-first traversal. Non-overlapping link root paths produce separately linked file sets.

.. include:: /_links_sphinx.r?st, with the one initial / instead of a relative or absolute path, will automatically search upward for the _links_xxx.r?st file (_sphinx is replaced by what is needed by the wanted target when the docs are generated).

Sphinx conf.py is augmented by configuration for Pandoc and Docutils. It should be where the input file is, or better at the project root to be usable with waf.

See the example project created with --rest/stpl/ipdt/over and the sources of the documentation of rstdoc.

`rstdcx` CLI¶

rstdcx is the same as rstdoc.

Without parameters: creates |substitution| links and .tags ctags for reST targets.

With two or three parameters: process file or dir to out file or dir through Pandoc, Sphinx, Docutils (third parameter):

html, docx, odt, pdf, … uses Pandoc.
rst_html, rst_odt, rst_pdf, … uses rst2html, …
sphinx_html, sphinx_pdf, … uses Sphinx. Sphinx provides a nice entry point via the sphinx bootstrap theme.

4th parameter onward become python defines usable in .stpl files.

Pdf output needs latex. Else you can make odt or docx and use

win: swriter.exe --headless --convert-to pdf Untitled1.odt
linux: lowriter --headless --convert-to pdf Untitled1.odt

Inkscape (.eps, .svg), Dot (.dot), Planuml (.uml), latex (.tex,.tikz) are converted to .png into ./_images or <updir>/_images or ‘.’. Any of the files can be a SimpleTemplate template (xxx.yyy.stpl).

Configuration is in conf.py or ../conf.py.

rstdoc --stpl|--rest|--ipdt|-over create sample project trees.

--stpl with .rest.stpl template files, --rest with only a doc folder with .rest files, --ipdt with inform-plan-do-test enhancement cycles --over with .rest files all over the project tree including symbolic links

Examples¶

Example folders (see wscript and Makefile there):

rstdoc --rest <folder> [--rstrest]
rstdoc --stpl <folder> [--rstrest]
rstdoc --ipdt <folder> [--rstrest]
rstdoc --over <folder> [--rstrest]

Use --rstrest to produce .rst for the main file, as .rest is not recognized by github/gitlab, who also don’t support file inclusion, so no need for two extension anyway.

Examples usages with the files generated by rstdoc --stpl tmp:

cd tmp/doc
rstdcx   #expand .stpl and produce .tag and _links_xxx files

#expand stpl and append substitutions (for simple expansion use ``stpl <file> .``)
rstdcx dd.rest.stpl - rest           # expand to stdout, appending dd.html substitutions, to pipe to Pandoc
rstdcx dd.rest.stpl - html.          # as before
rstdcx dd.rest.stpl - docx.          # expand to stdout, appending dd.docx substitutions, to pipe to Pandoc
rstdcx dd.rest.stpl - newname.docx.  # expand template, appending substitutions for target newname.docx
rstdcx dd.rest.stpl - html           # expand to stdout, already process through Pandoc to produce html on stdout
rstdcx dd.rest.stpl                  # as before
rstdcx sy.rest.stpl - rst_html       # expand template, already process through Docutils to produce html on stdout
stpl sy.rest.stpl | rstdcx - - sy.html. # appending sy.html substitutions, e.g. to pipe to Pandoc
stpl dd.rest.stpl | rstdcx - - dd.html  # appending tp.html substitutions and produce html on stdout via Pandoc
rstdcx dd.rest.stpl dd.rest          # expand into dd.rest, appending substitutions for target dd.html
rstdcx dd.rest.stpl dd.html html     # expand template, process through Pandoc to produce dd.html
rstdcx dd.rest.stpl dd.html          # as before
rstdcx dd.rest.stpl dd.html rst_html # expand template, already process through Docutils to produce dd.html
rstdcx dd.rest.stpl dd.docx          # expand template, process through Pandoc to produce dd.docx
rstdcx dd.rest.stpl dd.odt pandoc    # expand template, process through Pandoc to produce dd.odt
rstdcx dd.rest.stpl dd.odt           # as before
rstdcx dd.rest.stpl dd.odt rst_odt   # expand template, process through Docutils to produce dd.odt
rstdcx dd.rest.stpl dd.odt rst       # as before
rstdcx . build html                  # convert current dir to build output dir using pandoc
rstdcx . build sphinx_html           # ... using sphinx (if no index.rest, every file separately)

#Sphinx is not file-oriented
#but with rstdcx you need to provide the files to give Sphinx ``master_doc`` (normally: index.rest)
#Directly from ``.stpl`` does not work with Sphinx
rstdcx index.rest ../build/index.html sphinx_html   # via Sphinx the output directory must be different

#convert the graphics and place the into _images or <updir>/_images
#if no _images directory exists they will be placed into the same directory
rstdcx egcairo.pyg
rstdcx egdot.dot.stpl
rstdcx egeps.eps
rstdcx egother.pyg
rstdcx egplt.pyg
rstdcx egpygal.pyg
rstdcx egpyx.pyg
rstdcx egsvg.svg.stpl
rstdcx egtikz.tikz
rstdcx egtikz1.tikz
rstdcx eguml.uml

#Convert graphics to a png (even if _images directory exists):
rstdcx eguml.uml eguml.png

#Files to other files:

rstdoc dd.rest.stpl dd.rest
rstdoc dd.rest.stpl dd.html html
rstdoc dd.rest.stpl dd.html
rstdoc sr.rest.stpl sr.html rst_html
rstdoc dd.rest.stpl dd.docx
rstdoc dd.rest.stpl dd.odt pandoc
rstdoc dd.rest.stpl dd.odt
rstdoc sr.rest.stpl sr.odt rst_odt
rstdoc sr.rest.stpl sr.odt rst
rstdoc index.rest build/index.html sphinx_html

#Directories to other directories with out info:

rstdoc . build html
rstdoc . build sphinx_html

Grep with python re in .py, .rst, .rest, .stpl, .tpl:

rstdoc --pygrep inline

Grep for keyword lines containing ‘png’:

rstdoc --kw png

Default keyword lines:

.. {{{kw1,kw2
.. {kw1,kw2}
{{_ID3('kw1 kw2')}}
%__ID3('kw1 kw2')
:ID3: kw1 kw2

API¶

import rstdoc.dcx as dcx

The functions in dcx.py are available to the gen_xxx(lns,**kw) functions (dhy).

dcx.is_project_root_file:

def is_project_root_file(filename):

return filename=='.git' or filename=='waf' or filename=='Makefile' or filename.lower().startswith('readme')

Identifies the root of the project by a file name contained there.

dcx.DPI:

DPI = 600

Used for png creation.

dcx.g_config:

g_config = None

g_config can be used to inject a global config. This overrides the defaults and is overriden by an updir conf.py.

dcx.cmd:

def cmd(cmdlist, **kwargs):

Runs cmdlist via subprocess.run and return stdout. In case of problems RstDocError is raised.

param cmdlist:	command as list
param kwargs:	arguments forwarded to `subprocess.run()`

dcx.new_cwd:

@contextlib.contextmanager
def new_cwd(apth):

Use as:

with new_cwd(dir):
    #inside that directory

dcx.startfile:

def startfile(filepath):

Extends the Python startfile to non-Windows platforms

dcx.up_dir:

def up_dir(match,start=None):

Find a parent path producing a match on one of its entries. Without a match an empty string is returned.

param match:	a function returning a bool on a directory entry
param start:	absolute path or None
return:	directory with a match on one of its entries

>>> up_dir(lambda x: False)
''

dcx.tempdir:

def tempdir():

Make temporary directory and register it to be removed with atexit.

This can be used inside a .stpl file to create images from inlined images source, place them in temporary file, and include them in the final .docx or .odt.

dcx.run_inkscape:

def run_inkscape(infile,  outfile, dpi=DPI):

Uses inkscape commandline to convert to .png

param infile:	.svg, .eps, .pdf filename string (for list with actual .eps or .svg data use dcx.svgpng or dcx.epspng)
param outfile:	.png file name

dcx.rst_sphinx:

@infile_cwd
def rst_sphinx(
        infile, outfile, outtype=None, **config
        ):

Run Sphinx on infile.

param infile:	.txt, .rst, .rest filename
param outfile:	the path to the target file (not target directory)
param outtype:	html, latex,… or any other sphinx writer
param config:	keys from config_defaults

>>> olddir = os.getcwd()
>>> cd(dirname(__file__))
>>> cd('../doc')

>>> infile, outfile = ('index.rest',
... '../build/doc/sphinx_html/index.html')
>>> rst_sphinx(infile, outfile) #doctest: +ELLIPSIS
>>> exists(outfile)
True

>>> infile, outfile = ('dd.rest',
... '../build/doc/sphinx_html/dd.html')
>>> rst_sphinx(infile, outfile) #doctest: +ELLIPSIS
>>> exists(outfile)
True

>>> infile, outfile = ('dd.rest',
... '../build/doc/sphinx_latex/dd.tex')
>>> rst_sphinx(infile, outfile) #doctest: +ELLIPSIS
>>> exists(outfile)
True

>>> cd(olddir)

dcx.g_include:

g_include = []

One can append paths to rstdoc.dcx.g_include for stpl expansion or finding other files.

dcx.rst_pandoc:

@infile_cwd
def rst_pandoc(
        infile, outfile, outtype, **config
        ):

Run Pandoc on infile.

param infile:	.txt, .rst, .rest filename
param outfile:	the path to the target document
param outtype:	html,…
param config:	keys from config_defaults

dcx.rst_rst2:

@infile_cwd
def rst_rst2(
        infile, outfile, outtype, **config
        ):

Run the rst2xxx docutils fontend tool on infile.

param infile:	.txt, .rst, .rest filename
param outfile:	the path to the target document
param outtype:	html,…
param config:	keys from config_defaults

dcx.PageBreakHack:

def PageBreakHack(destination_path):

This introduces a PageBreak style into content.xml to allow the following raw page break of opendocument odt:

.. raw:: odt

    <text:p text:style-name="PageBreak"/>

This is no good solution, as it introduces an empty line at the top of the new page.

Unfortunately the following does not work with or without text:use-soft-page-breaks="true"

.. for docutils
.. raw:: odt

    <text:p text:style-name="PageBreak"/>

.. for pandoc
.. raw:: opendocument

    <text:p text:style-name="PageBreak"/>

According to C066363e.pdf it should work.

See utility.rst.tpl in the --stpl created example project tree.

dcx.svgpng:

@png_post_process_if_any
@normoutfile
@readin
def svgpng(infile, outfile=None, *args, **kwargs):

Converts a .svg file to a png file.

param infile:	a .svg file name or list of lines
param outfile:	if not provided the input file with new extension `.png` in `./_images`, `<updir>/_images` or parallel to infile.

dcx.texpng:

@png_post_process_if_any
@partial(in_temp_if_list, suffix='.tex')
@infile_cwd
def texpng(infile, outfile=None, *args, **kwargs):

Latex has several graphic packages, like

tikz
chemfig

that can be converted to .png with this function.

For .tikz file use dcx.tikzpng.

param infile:	a .tex file name or list of lines (provide outfile in the latter case)
param outfile:	if not provided, the input file with `.png` in `./_images`, `<updir>/_images` or parallel to infile.

dcx.tikzpng:

tikzpng = normoutfile(readin(_tikzwrap(_texwrap(texpng))))

Converts a .tikz file to a png file.

See dcx.texpng.

dcx.dotpng:

@png_post_process_if_any
@partial(in_temp_if_list, suffix='.dot')
@infile_cwd
def dotpng(
        infile,
        outfile=None,
        *args,
        **kwargs
        ):

Converts a .dot file to a png file.

param infile:	a .dot file name or list of lines (provide outfile in the latter case)
param outfile:	if not provided the input file with new extension `.png` in `./_images`, `<updir>/_images` or parallel to infile.

dcx.umlpng:

@png_post_process_if_any
@partial(in_temp_if_list, suffix='.uml')
@infile_cwd
def umlpng(
        infile,
        outfile=None,
        *args,
        **kwargs
        ):

Converts a .uml file to a png file.

param infile:	a .uml file name or list of lines (provide outfile in the latter case)
param outfile:	if not provided the input file with new extension `.png` in `./_images`, `<updir>/_images` or parallel to infile.

dcx.epspng:

@png_post_process_if_any
@partial(in_temp_if_list, suffix='.eps')
@infile_cwd
def epspng(
        infile,
        outfile=None,
        *args,
        **kwargs):

Converts an .eps file to a png file using inkscape.

param infile:	a .eps file name or list of lines (provide outfile in the latter case)
param outfile:	if not provided the input file with new extension `.png` in `./_images`, `<updir>/_images` or parallel to infile.

dcx.pygpng:

@png_post_process_if_any
@normoutfile
@readin
@infile_cwd
def pygpng(
        infile, outfile=None, *args,
        **kwargs
        ):

Converts a .pyg file to a png file.

.pyg contains python code that produces a graphic. If the python code defines a to_svg or a save_to_png function, then that is used. Else the following is tried

pyx.canvas.canvas from the pyx library or
svgwrite.drawing.Drawing from the svgwrite library or
cairocffi.Surface from cairocffi
matplotlib. If matplotlib.pyplot.get_fignums()>1 the figures result <name><fignum>.png

param infile:	a .pyg file name or list of lines (provide outfile in the latter case)
param outfile:	if not provided the input file with new extension `.png` in `./_images`, `<updir>/_images` or parallel to infile.

dcx.pygsvg:

@readin
@infile_cwd
def pygsvg(infile, *args, **kwargs):

Converts a .pyg file or according python code to an svg string.

.pyg contains python code that produces an SVG graphic. Either there is a to_svg() function or the following is tried

io.BytesIO containing SVG, e.g via cairo.SVGSurface(ioobj,width,height)
io.StringIO containing SVG
object with attribute _repr_svg_
svgwrite.drawing.Drawing from the svgwrite library or
cairocffi.SVGSurface from cairocffi
matplotlib.

param infile:	a .pyg file name or list of lines

dcx.svgembed:

def svgembed(
        pyg_or_svg, outinfo, *args, **kwargs
        ):

If outinfo ends with html, SVG is embedded. Else the SVG is converted to a temporary image file and included in the DOCX or ODT zip.

dcx.pngembed:

def pngembed(
        pyg_or_pngfile, outinfo, *args, **kwargs
        ):

If outinfo ends with html, the PNG is embedded. Else the PNG is included in the DOCX or ODT zip.

dcx.dostpl:

@infile_cwd
def dostpl(
        infile,
        outfile=None,
        lookup=None,
        **kwargs
        ):

Expands an .stpl file.

The whole rstdoc.dcx namespace is forwarded to the template code.

.stpl is unrestrained python:

e.g. one can create temporary images, which are then included in the final .docx of .odt See dcx.tempdir.

param infile:	a .stpl file name or list of lines
param outfile:	if not provided the expanded is returned
param lookup:	lookup paths can be absolute or relative to infile

>>> infile = ['hi {{2+3}}!']
>>> dostpl(infile)
['hi 5!']

dcx.dorst:

def dorst(
        infile,
        outfile=io.StringIO,
        outinfo=None,
        fn_i_ln=None,
        **kwargs
        ):

Default interpreted text role is set to math. The link lines are added to the rest file or rst lines

param infile:	a .rest, .rst, .txt file name or list of lines
param outfile:	None and ‘-’ mean standard out. If io.StringIO, then the lines are returned. `\|xxx\|` substitutions for reST link targets in infile are appended if no `_links_sphinx.rst` there
param outinfo:	specifies the tool to use. `html`, `docx`, `odt`,… via pandoc `sphinx_html`,… via sphinx `rst_html`,… via rst2xxx frontend tools General format of outinfo: [infile/][tgtfile.]docx[.] `infile` is used, if the function infile param are lines. `tgtfile` is target file used in links. `tgtfile` is the target file to create. A final dot tells not to create the target file. This is of use in the command line if piping a file to rstdoc then to pandoc. The doc will only be generated by pandoc, but links need to know the doc to link to already before that.
param fn_i_ln:	`(fn, i, ln)` of the `.stpl` with all stpl includes sequenced (used by dcx.convert)

>>> olddir = os.getcwd()
>>> cd(dirname(__file__))
>>> cd('../doc')

>>> dorst('dd.rest') #doctest: +ELLIPSIS
['.. default-role:: math\n', ...

>>> dorst('ra.rest.stpl') #doctest: +ELLIPSIS
['.. default-role:: math\n', ...

>>> dorst(['hi there']) #doctest: +ELLIPSIS
['.. default-role:: math\n', '\n', 'hi there\n', ...

>>> dorst(['hi there'], None,'html') #doctest: +ELLIPSIS
<!DOCTYPE html>
...

>>> drst=lambda x,y: dorst(x,y,None,pandoc_doc_optref={'docx':'--reference-doc doc/reference.'+y.split('.')[1]})
>>> dorst('ra.rest.stpl','ra.docx') #doctest: +ELLIPSIS
>>> exists('ra.docx')
True
>>> rmrf('ra.docx')
>>> exists('ra.docx')
False
>>> rmrf('ra.rest.stpl.rest')
>>> exists('ra.rest.stpl.rest')
False

>>> dorst(['hi there'],'test.html') #doctest: +ELLIPSIS
>>> exists('test.html')
True
>>> rmrf('test.html')
>>> exists('test.html')
False
>>> rmrf('rest.rest.rest')
>>> exists('rest.rest.rest')
False

>>> dorst(['hi there'],'test.odt','rst') #doctest: +ELLIPSIS
>>> exists('rest.rest.rest')
True
>>> rmrf('rest.rest.rest')
>>> exists('rest.rest.rest')
False
>>> exists('test.odt')
True
>>> rmrf('test.odt')
>>> exists('test.odt')
False
>>> cd(olddir)

dcx.convert:

def convert(
        infile,
        outfile=io.StringIO,
        outinfo=None,
        **kwargs
        ):

Converts any of the known files.

Stpl files are forwarded to the next converter.

The main job is to normalized the input params, because this is called from dcx.main and via Python. It forwards to the right converter.

Examples:

>>> olddir = os.getcwd()
>>> cd(dirname(__file__))
>>> cd('../doc')

>>> convert([' ','   hi {{2+3}}!'], outinfo='rest')
['   .. default-role:: math\n', '\n', ' \n', '   hi 5!\n', '\n']

>>> convert([' ','   hi {{2+3}}!'])  #doctest: +ELLIPSIS
['<!DOCTYPE html>\n', ...]
>>> rmrf('rest.rest.rest')

>>> infile, outfile, outinfo = ([
... "newpath {{' '.join(str(i)for i in range(4))}} rectstroke showpage"
... ],'tst.png','eps')
>>> 'tst.png' in convert(infile, outfile, outinfo) #doctest: +ELLIPSIS
True
>>> exists('tst.png')
True
>>> rmrf('tst.png')
>>> exists('tst.png')
False

>>> convert('ra.rest.stpl') #doctest: +ELLIPSIS
['<!DOCTYPE html>\n', ...

>>> cnvrt=lambda x,y: convert(x,y,None,pandoc_doc_optref={'docx':'--reference-doc doc/reference.'+y.split('.')[1]})
>>> cnvrt('ra.rest.stpl','ra.docx')
>>> exists('ra.rest.rest')
True
>>> rmrf('ra.rest.rest')
>>> exists('ra.rest.rest')
False
>>> exists('ra.docx')
True
>>> rmrf('ra.docx')
>>> exists('ra.docx')
False

>>> convert('dd.rest', None,'html') #doctest: +ELLIPSIS
<!DOCTYPE html>
...
>>> exists('dd.rest.rest')
True
>>> rmrf('dd.rest.rest')
>>> exists('dd.rest.rest')
False
>>> cd(olddir)

param infile:	any of `.tikz`, `.svg`, `.dot`, `.uml`, `.eps`, `.pyg` or else stpl is assumed. Can be list of lines, too.
param outfile:	`-` means standard out, else a file name, or None for automatic (using outinfo), or io.StringIO to return lines instead of stdout
param outinfo:	`html`, `sphinx_html`, `docx`, `odt`, `file.docx`,… interpet input as rest, else specifies graph type

dcx.convert_in_tempdir:

convert_in_tempdir = in_temp_if_list(infile_cwd(convert))

Same as dcx.convert, but creates temporary dir for a list of lines infile argument.

>>> tmpfile = convert_in_tempdir("""digraph {
... %for i in range(3):
...    "From {{i}}" -> "To {{i}}";
... %end
...    }""".splitlines(), outinfo='dot')
>>> stem_ext(tmpfile)[1]
'.png'
>>> tmpfile = convert_in_tempdir("""
... This is re{{'st'.upper()}}
...
... .. _`xx`:
...
... xx:
...     text
...
... """.splitlines(), outinfo='rst_html')
>>> stem_ext(tmpfile)[1]
'.html'

dcx.rindices:

def rindices(regex, lns):

Return the indices matching the regular expression regex.

param regex:	regular expression string or compiled
param lns:	lines

>>> lns=['a','ab','b','aa']
>>> [lns[i] for i in rindices(r'^a\w*', lns)]==['a', 'ab', 'aa']
True

dcx.rlines:

def rlines(regex, lns):

Return the lines matched by regex.

param regex:	regular expression string or compiled
param lns:	lines

dcx.doc_parts:

def doc_parts(
        lns,
        relim=r"^\s*r?'''([\w.:]*)\s*\n*$",
        reid=r"\s(\w+)[(:]|(\w+)\s\=",
        reindent=r'[^#/\s]',
        signature=None,
        prefix=''
        ):

doc_parts() yields doc parts delimited by relim regular expression possibly with id, if reid matches

If start and stop differ use regulare expression | in relim.

There is no empty line between doc string and preceding code lines that should be included.
There is no empty line between doc string and succeeding code lines that should be included.
Included code lines end with an empty line.

In case of __init__() the ID can come from the class line and the included lines can be those of __init__(), if there is no empty line between the doc string and class above as well as _init__() below.

If the included code comes only from one side of the doc string, have an empty line at the other side.

Immediately after the initial doc string marker there can be a prefix, e.g. classname..

param signature:
param lns:	list of lines
param relim:	regular expression marking lines enclosing the documentation. The group is a prefix.
param reid:	extract id from preceding or succeeding non-empty lines
param reindent:	determines start of text
	if signature language is given the preceding or succeeding lines will be included
param prefix:	prefix to make id unique, e.g. module name. Include the dot.

>>> with open(__file__) as f:
...     lns = f.readlines()
...     docparts = list(doc_parts(lns, signature='py'))
...     doc_parts_line = rlines('doc_parts', docparts)
>>> doc_parts_line[1]
':doc_parts:\n'

dcx.rstincluded:

@_memoized
def rstincluded(
        fn,
        paths=(),
        withimg=False,
        withrest=False
        ):

Yield the files recursively included from an RST file.

param fn:	file name without path
param paths:	paths where to look for fn
param withimg:	also yield image files, not just other RST files
param withrest:	rest files are not supposed to be included

>>> olddir = os.getcwd()
>>> cd(dirname(__file__))
>>> list(rstincluded('ra.rest',('../doc',)))
['ra.rest.stpl', '_links_sphinx.rst']
>>> list(rstincluded('sr.rest',('../doc',)))
['sr.rest', '_links_sphinx.rst']
>>> list(rstincluded('meta.rest',('../doc',)))
['meta.rest', 'files.rst', '_traceability_file.rst', '_links_...']
>>> 'dd.rest' in list(rstincluded(
... 'index.rest',('../doc',), False, True))
True
>>> cd(olddir)

dcx.pair:

def pair(alist, blist, cmp):

pair two sorted lists where the second must be at least as long as the first

param alist:	first list
param blist:	second list longer or equal to alist
param cmp:	compare function

>>> alist=[1,2,4,7]
>>> blist=[1,2,3,4,5,6,7]
>>> cmp = lambda x,y: x==y
>>> list(pair(alist,blist,cmp))
[(1, 1), (2, 2), (None, 3), (4, 4), (None, 5), (None, 6), (7, 7)]

>>> alist=[1,2,3,4,5,6,7]
>>> blist=[1,2,3,4,5,6,7]
>>> cmp = lambda x, y: x==y
>>> list(pair(alist, blist, cmp))
[(1, 1), (2, 2), (3, 3), (4, 4), (5, 5), (6, 6), (7, 7)]

dcx.gen:

def gen(
        source,
        target=None,
        fun=None,
        **kw
        ):

Take the gen_[fun] functions enclosed by #def gen_[fun](lns,**kw) to create a new file.

param source:	either a list of lines or a path to the source code
param target:	either save to this file or return the generated documentation
param fun:	use `#gen_<fun>(lns,**kw):` to extract the documentation
param kw:	kw arguments to the `gen_<fun>()` function

>>> source=[i+'\\n' for i in """
...        #def gen(lns,**kw):
...        #  return [l.split('#@')[1] for l in rlines(r'^\s*#@', lns)]
...        #def gen
...        #@some lines
...        #@to extract
...        """.splitlines()]
>>> [l.strip() for l in gen(source)]
['some lines', 'to extract']

dcx.parsegenfile:

def parsegenfile(genpth):

Parse the file genpth which is either

python code or
has format

sourcefile | targetfile | suffix | kw paramams or {}

suffix refers to gen_<suffix>.

The yields are used for the dcx.gen function.

param genpth:	path to gen file

dcx.RstFile.__init__:

class RstFile:
    def __init__(self, reststem, doc, tgts, lnks, nlns):

Contains the targets for a .rst or .rest file.

param reststem:	rest file this doc belongs to (without extension)
param doc:	doc belonging to reststem, either included or itself (.rest, .rst, .stpl)
param tgts:	list of Tgt objects yielded by dcx.RstFile.make_tgts.
param lnks:	list of (line index, target name (`\|target\|`)) tuples
param nlns:	number of lines of the doc

dcx.RstFile.make_tgts:

@staticmethod
def make_tgts(
        lns,
        doc,
        counters=None,
        fn_i_ln=None
        ):

Yields ((line index, tag address), target, link name) of lns of a restructureText file. For a .stpl file the linkname comes from the generated RST file.

param lns:	lines of the document
param doc:	the rst/rest document for tags
param counters:	if None, the starts with {“.. figure”:1,”.. math”:1,”.. table”:1,”.. code”:1}
fn_i_ln:	(fn, i, ln) of the .stpl with all stpl includes sequenced

dcx.links_and_tags:

def links_and_tags(
    scanroot='.'
    ):

Creates _links_xxx.rst files and a .tags.

param scanroot:	directory for which to create links and tags

>>> olddir = os.getcwd()
>>> cd(dirname(__file__))
>>> rmrf('../doc/_links_sphinx.rst')
>>> '_links_sphinx.rst' in ls('../doc')
False

>>> links_and_tags('../doc')
>>> '_links_sphinx.rst' in ls('../doc')
True
>>> cd(olddir)

dcx.grep:

def grep(
      regexp=rexkw,
      dir=None,
      exts=set(['.rst','.rest','.stpl','.tpl','.adoc','.md','.wiki','.py','.jl','.lua','.tex',
                '.js', '.h','.c','.hpp','.cpp','.java','.cs','.vb','.r','.sh','.vim','.el',
                '.php','.sql','.swift','.go','.rb','.m','.pl','.rs','.f90','.dart','.bib',
                '.yml','.mm','.d','.lsp','.kt','.hs','.lhs','.ex','.scala','.clj']),
      **kwargs
):

Uses python re to find regexp and return [(file,1-based index,line),...] in dir (default: os.getcwd()) for exts files

param regexp:	default is ‘^s*.. {’
param dir:	default is current dir
param exts:	the extension of files searched

def yield_with_kw (kws, fn_ln_kw=None, **kwargs):

Find keyword lines in fn_ln_kw list or using grep(), that contain the keywords in kws.

Keyword line are either of:

.. {{{kw1,kw2
.. {kw1,kw2}
{{_ID3('kw1 kw2')}}
%__ID3('kw1 kw2')
:ID3: kw1 kw2

.. can also be two comment chars of popular programming languages. This is due to dcx.rexkw, which you can change. See also dcx.grep() for the keyword parameters.

param kws:	string will be split by non-chars
param fn_ln_kw:	list of (file, line, keywords) tuples or `regexp` for grep()

>>> list(yield_with_kw('a',[('a/b',1,'a b'),('c/d',1,'c d')]))
[(0, ['a/b', 1, 'a b'])]
>>> list(yield_with_kw('a c',[('a/b',1,'a b'),('c/d',1,'c d')]))
[]
>>> list(yield_with_kw('a',[('a/b',1,'a b'),('c/d',1,'a c d')]))
[(0, ['a/b', 1, 'a b']), (1, ['c/d', 1, 'a c d'])]
>>> kwargs={'dir':normjoin(dirname(__file__),'../test/fixtures')}
>>> kws = 'svg'
>>> len(list(yield_with_kw(kws,**kwargs)))
6
>>> kws = 'png'
>>> len(list(yield_with_kw(kws,**kwargs)))
7

dcx.Counter.__init__:

class Counter:
    def __init__(self, before_first=0):

Counter object.

param before_first:
	first-1 value

>>> myc = Counter()
>>> myc()
1
>>> myc()
2

dcx.x:

gpdtid = pdtid
def pdtAAA(pdtfile,dct,pdtid=pdtid,
            pdtfileid=lambda x:'ipdt'[int(x[0])]):

pdtAAA is for use in an .stpl document:

% pdtAAA(__main_file__,globals())

See the example generated with:

rstdoc --ipdt

param pdtfileid:
param pdtfile:	file path of pdt
param dct:	dict to take up the generated defines
param pdtid:	function returning the ID for the `pdt` cycle or regular expression with group for full path or regular expression for just the base name without extension (`pdtok`)
	extracts/maps a file base name to one of the letters ipdt. E.g. to have the files in order one could name them {0,1,2,3}.rest.stpl, and map each to one of ‘ipdt’.

A pdt is a project enhancement cycle with its own documentation. pdt stands for

plan: why
do: specification
test: tests according specification

Additionally there should be an

inform: non-technical purpose for or from external people.

There can also be only the inform document, if the pdt item is only informative.

The repo looks like this (preferred):

project repo
    pdt
        ...
        AAA
            i*.rest.stpl
            p*.rest.stpl
            d*.rest.stpl
            t*.rest.stpl

or:

project repo
    pdt
        ...
        AAA.rst.stpl

In the first case, the UID starts with {i,p,d,t}AAA. This is useful to trace related items by their plan-do-test-aspect.

rstfromdocx¶

rstfromdocx: shell command

fromdocx: rstdoc module

Convert DOCX to RST in a subfolder of current dir, named after the DOCX file. It also creates conf.py, index.py and Makefile and copies dcx.py into the folder.

See rstdcx for format conventions for the RST.

There are options to post-process through:

--listtable (--join can be provided)
--untable
--reflow (--sentence True,  --join 0)
--reimg

rstfromdocx -lurg combines all of these.

To convert more DOCX documents into the same RST documentation folder, proceed like this:

rename/copy the original DOCX to the name you want for the .rest file
run rstfromdocx -lurg doc1.docx; instead of -lurg use your own options
check the output in the doc1 subfolder
repeat the previous 2 steps with the next DOCX files
create a new folder, e.g. doc
merge all other folders into that new folder

fromdocx.docx_rst_5 creates 5 different rst files with different postprocessing.

See rstreflow for an alternative proceeding.

API¶

import rstdoc.fromdocx as fromdocx

fromdocx.extract_media:

def extract_media(adocx):

extract media files from a docx file to a subfolder named after the docx

param adocx:	docx file name

fromdocx.main:

def main(**args):

This corresponds to the rstfromdocx shell command.

param args:	Keyword arguments. If empty the arguments are taken from `sys.argv`.

listtable, untable, reflow, reimg default to False.

returns: The file name of the generated file.

fromdocx.docx_rst_5:

def docx_rst_5(docx ,rename ,sentence=True):

Creates 5 rst files:

without postprocessing: rename/rename.rest
with listtable postprocessing: rename/rename_l.rest
with untable postprocessing: rename/rename_u.rest
with reflow postprocessing: rename/rename_r.rest
with reimg postprocessing: rename/rename_g.rest

param docx:	the docx file name
param rename:	the new name to give to the converted files (no extension)
param sentence:	split sentences into new lines (reflow)

rstlisttable¶

rstlisttable: shell command

listable: rstdoc module

Convert RST grid tables to list-tables.

Convert grid tables in a file to list-tables. The result is output to stdout:
```
$ listtable.py input.rst
```

Convert several files:

$ listtable.py input1.rst input2.rst
$ listtable.py *.rst

Use pipe (but cat might not keep the encoding):

$ cat in.rst | listtable.py -  | untable.py - > out.rst

Options¶

-j, --join

e.g.002. Join method per column: 0=””.join; 1=” “.join; 2=”\n”.join

API¶

import rstdoc.listtable as listtable

listtable.row_to_listtable:

def row_to_listtable(
        row ,colwidths ,withheader ,join ,indent ,tableend
    ):

This is the default process_row parameter of listtable.gridtable.

param colwidths:
param row:	list of cells for the row
	The widths of the columns
param withheader:
	produce :header-:param rows: 1
param join:	0,1,2 telling how to combine the lines of a cell

0 = without space
1 = with space
2 = keep multi-line

param indent:	indentation of the table
param tableend:	True, if end of table

listtable.gridtable:

def gridtable(
        data ,join='012' ,process_row=row_to_listtable
    ):

Convert grid table to list table with same column number throughout. See listtable.row_to_listtable.

param process_row:
param data:	from file.readlines() or str.splitlines(True)
param join:	join column 0 without space, column 1 with space and leave the rest as-is
	creates a list-table entry for the row

listtable.main:

def main(**args):

This corresponds to the rstlisttable shell command.

param args:	Keyword arguments. If empty the arguments are taken from `sys.argv`.

rstfile is the file name

in_place defaults to False

join defaults to “012”

rstuntable¶

rstuntable: shell command

untable: rstdoc module

Convert tables of following format to paragraphs with an ID. The ‘-’ in ID is removed and the ID is made lower case. Bold is removed.

ID-XY-00	text goes here
ID-XY-01	text again goes here

If the first entry does contain no word chars or spaces between words, then the table stays. For a different behavior replace paragraph23.

A file produced from a docx using pandoc or fromdocx.py will need a pre-processing via rstlisttable to convert grid tables to list-table tables. This is done in one step with rstfromdocx -lu doc.rst.

API¶

import rstdoc.untable as untable

untable.paragraph23:

def paragraph23(row, nColumns, org, islast, withheader):

For process_row parameter of untable.

For a table of 2 or 3 columns, transform to text. The first column must hold only one line for an ID.

If not transformed to paragraph, then the orginal text (org) is yielded.

param withheader:
param row:	list of strings representing the row
param nColumns:	number of columns in the table
param org:	orginal text
param islast:	this call is with the last table entry
	the table has a header line

untable.untable:

def untable(lns, process_row=paragraph23):

Transform a RST list-table to normal paragraphs. The table is supposed to have this format:

The first column holds an ID.
Optionally the second column holds keywords.
The last column holds the details.

param process_row:
param lns:	list of strings
	called for each row to transform to paragraph

untable.main:

def main(**args):

This corresponds to the rstuntable shell command.

param args:	Keyword arguments. If empty the arguments are taken from `sys.argv`.

rstfile is the file name

in_place defaults to False

rstreflow¶

rstreflow: shell command

reflow: rstdoc module

Reflow tables and paragraphs in a rst document produced from a docx.

Post-process a docx in this order:

rstfromdocx doc.docx
rstlisttable doc/doc.rst > doc/tmp.rst
rstuntable doc/tmp.rst > doc/tmp1.rst
rstreflow doc/tmp1.rst > doc/tmp2.rst
rstreimg doc/tmp2.rst > doc/tmp3.rst
rm doc/doc.rst
mv doc/tmp3.rst doc/doc.rst
rm doc/tmp*

Check the intermediate results.

Else one can also do inplace:

rstfromdocx doc.docx
rstlisttable -i doc/doc.rst
rstuntable -i doc/doc.rst
rstreflow -i doc/doc.rst
rstreimg -i doc/doc.rst

Note

DOCX to RST using Pandoc

rstfromdocx -lurg doc.rst converts a docx to RST and does all the post-processing in one step.

It is adviced, though, to compare the output with the original and do some manual corrections here and there.

API¶

import rstdoc.reflow as reflow

reflow.reflowparagraph:

def reflowparagraph(p, sentence=False):

Reflow a paragaph using textwarp.wrap. Possibly split sentences.

param p:	paragraph
param sentence:	if True lines are split at the end of the sentence

reflow.reflowparagraphs:

def reflowparagraphs(lns, sentence=False):

Reflow paragraphs using reflow.reflowparagraph.

param lns:	lines from rst file
param sentence:	if True lines are split at the end of the sentence

reflow.nostrikeout:

def nostrikeout(lns):

Removes [strikeout:xxx]

param lns:	lines from rst file

reflow.rmextrablankline:

def rmextrablankline(lns):

Remove excessive blank lines.

param lns:	lines from rst file

reflow.no3star:

def no3star(lns):

Removes three stars, as they are not supported by docutils.

param lns:	lines from rst file

reflow.noblankend:

def noblankend(lns):

Removes blanks at the end of the line.

param lns:	lines from rst file

reflow.reflowrow:

class reflowrow():

This replaces listtable.row_to_listtable in listtable.gridtable to reflow a grid table.

reflow.reflow:

def reflow(lns, join='1', sentence=False):

Combines all rst corrections of this file.

param lns:	lines from rst file
param join:	0 no space, 1 with space, 2 keep as-is
param sentence:	if True lines are split at the end of the sentence

reflow.main:

def main(**args):

This corresponds to the rstreflow shell command.

param args:	Keyword arguments. If empty the arguments are taken from `sys.argv`.

rstfile is the file name

in_place defaults to False

rstreimg¶

rstreimg: shell command

reimg: rstdoc module

Reimg renames the images in the rst file and the files themselves. It uses part of the document name and a number as new names.

This is useful, if more RST documents converted from DOCX should be combined in one directory and the names of the images have no meaning (image13,…).

API¶

import rstdoc.reimg as reimg

reimg.reimg:

def reimg(data, prefix):

This renames all the images in the rst file converted from docx, to avoid

images having strange names
collision of image names from different docx

param data:	rst file read by f.read()
param prefix:	string prefix for images, should be derived from docx file name

reimg.main:

def main(**args):

This corresponds to the rstreimg shell command.

param args:	Keyword arguments. If empty the arguments are taken from `sys.argv`.

rstfile is the file name

in_place defaults to False

rstretable¶

rstretable: shell command

retable: rstdoc module

Transforms list tables to grid tables.

This file also contains the code from the Vim plugin vim-rst-tables-py3, plus some little fixes. rstdoc is used by the Vim plugin vim_py3_rst , which replaces vim-rst-tables-py3.

API¶

import rstdoc.retable as retable

retable.title_some:

title_some = """=-^"'`._~+:;,"""

The rst title order was partly taken from https://github.com/jimklo/atom-rst-snippets then converted to http://documentation-style-guide-sphinx.readthedocs.io/en/latest/style-guide.html

retable.reformat_table:

def reformat_table(lines, row=0, col=0, withheader=0):

Create or reformat a grid table in lines. The table is delimited by emtpy lines starting from (row,col).

param withheader:
param lines:	list of strings
param row:	of cursor position,
param col:	… as only the lines delimited by an empty line are used
	user the first line as table header

retable.create_rst_table:

def create_rst_table(data, withheader=0):

Create a rst table from data

Example:

>>> lns=[['one','two','three'],[1,2,3]]
>>> create_rst_table(lns)
'+-----+-----+-------+\n| one | two | three |\n+-----+-----+-------+\n| 1   | 2   | 3     |\n+-----+-----+-------+'

param data:	list of list of data

retable.reflow_table:

def reflow_table(lines, row=0, col=0):

Adapt an existing table to the widths of the first line. The table is delimited by emtpy lines starting from (row,col).

lines: list of strings row: of cursor position, col: … as only the lines delimited by an empty line are considered

retable.re_title:

def re_title(lines, row=0, col=0, down=0):

Adjust the under- or overline of a title.

param lines:	list of lines
param row:	of cursor position,
param col:	… as only the lines delimited by an empty line are considered
param down:	>0down, <0up

>>> lines="""\
...   ###########
...       title
...   ###########
...   """.splitlines()
>>> re_title(lines)
>>> lines
['      #####', '      title', '      #####', '  ']

retable.retable:

def retable(lns):

Transform listtable to grid table. Yield the resulting lines.

param lns:	list of strings

retable.main:

def main(**args):

This corresponds to the rstretable shell command.

param args:	Keyword arguments. If empty the arguments are taken from `sys.argv`.

rstfile is the file name

in_place defaults to False

param sampletype:
param rootfldr:	directory name that becomes root of the sample tree
	either ‘ipdt’ or ‘stpl’ for templated sample trees, or ‘rest’ or ‘over’ for non-templated

rstdoc¶

Purpose¶

rstdoc(1) Version 1.8.2 | rstdoc¶

rstdcx¶

Conventions¶

rstdcx CLI¶

Examples¶

API¶

rstfromdocx¶

API¶

rstlisttable¶

Options¶

API¶

rstuntable¶

API¶

rstreflow¶

API¶

rstreimg¶

API¶

rstretable¶

API¶

`rstdcx` CLI¶