Skip to content

Commit

Permalink
Merge https://github.com/angavrilov/dfhack
Browse files Browse the repository at this point in the history
  • Loading branch information
@peterix
peterix committed Jun 22, 2012
2 parents 6199d69 + 0e58290 commit 59fb4da
Show file tree
Hide file tree
Showing 65 changed files with 3,327 additions and 809 deletions.
23 changes: 22 additions & 1 deletion COMPILE.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -152,10 +152,13 @@ Valid and useful build types include 'Release', 'Debug' and
================================
Using the library as a developer
================================
Currently, the only way to use the library is to write a plugin that can be loaded by it.

Currently, the most direct way to use the library is to write a plugin that can be loaded by it.
All the plugins can be found in the 'plugins' folder. There's no in-depth documentation
on how to write one yet, but it should be easy enough to copy one and just follow the pattern.

Other than through plugins, it is possible to use DFHack via remote access interface, or by writing Lua scripts.

The most important parts of DFHack are the Core, Console, Modules and Plugins.

* Core acts as the centerpiece of DFHack - it acts as a filter between DF and SDL and synchronizes the various plugins with DF.
Expand All @@ -171,6 +174,24 @@ The main license is zlib/libpng, some bits are MIT licensed, and some are BSD li
Feel free to add your own extensions and plugins. Contributing back to
the dfhack repository is welcome and the right thing to do :)

DF data structure definitions
=============================

DFHack uses information about the game data structures, represented via xml files in the library/xml/ submodule.

Data structure layouts are described in files following the df.*.xml name pattern. This information is transformed by a perl script into C++ headers describing the structures, and associated metadata for the Lua wrapper. These headers and data are then compiled into the DFHack libraries, thus necessitating a compatibility break every time layouts change; in return it significantly boosts the efficiency and capabilities of DFHack code.

Global object addresses are stored in symbols.xml, which is copied to the dfhack release package and loaded as data at runtime.

Remote access interface
=======================

DFHack supports remote access by exchanging Google protobuf messages via a TCP socket. Both the core and plugins can define remotely accessible methods. The ``dfhack-run`` command uses this interface to invoke ordinary console commands.

Currently the supported set of requests is limited, because the developers don't know what exactly is most useful.

Protocol client implementations exist for Java and C#.

Contributing to DFHack
======================

Expand Down
33 changes: 24 additions & 9 deletions Compile.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -334,10 +334,12 @@ <h1 class="title">Building DFHACK</h1>
</li>
<li><a class="reference internal" href="#build-types" id="id12">Build types</a></li>
<li><a class="reference internal" href="#using-the-library-as-a-developer" id="id13">Using the library as a developer</a><ul>
<li><a class="reference internal" href="#contributing-to-dfhack" id="id14">Contributing to DFHack</a><ul>
<li><a class="reference internal" href="#coding-style" id="id15">Coding style</a></li>
<li><a class="reference internal" href="#how-to-get-new-code-into-dfhack" id="id16">How to get new code into DFHack</a></li>
<li><a class="reference internal" href="#memory-research" id="id17">Memory research</a></li>
<li><a class="reference internal" href="#df-data-structure-definitions" id="id14">DF data structure definitions</a></li>
<li><a class="reference internal" href="#remote-access-interface" id="id15">Remote access interface</a></li>
<li><a class="reference internal" href="#contributing-to-dfhack" id="id16">Contributing to DFHack</a><ul>
<li><a class="reference internal" href="#coding-style" id="id17">Coding style</a></li>
<li><a class="reference internal" href="#how-to-get-new-code-into-dfhack" id="id18">How to get new code into DFHack</a></li>
<li><a class="reference internal" href="#memory-research" id="id19">Memory research</a></li>
</ul>
</li>
</ul>
Expand Down Expand Up @@ -470,9 +472,10 @@ <h1><a class="toc-backref" href="#id12">Build types</a></h1>
</div>
<div class="section" id="using-the-library-as-a-developer">
<h1><a class="toc-backref" href="#id13">Using the library as a developer</a></h1>
<p>Currently, the only way to use the library is to write a plugin that can be loaded by it.
<p>Currently, the most direct way to use the library is to write a plugin that can be loaded by it.
All the plugins can be found in the 'plugins' folder. There's no in-depth documentation
on how to write one yet, but it should be easy enough to copy one and just follow the pattern.</p>
<p>Other than through plugins, it is possible to use DFHack via remote access interface, or by writing Lua scripts.</p>
<p>The most important parts of DFHack are the Core, Console, Modules and Plugins.</p>
<ul class="simple">
<li>Core acts as the centerpiece of DFHack - it acts as a filter between DF and SDL and synchronizes the various plugins with DF.</li>
Expand All @@ -485,18 +488,30 @@ <h1><a class="toc-backref" href="#id13">Using the library as a developer</a></h1
The main license is zlib/libpng, some bits are MIT licensed, and some are BSD licensed.</p>
<p>Feel free to add your own extensions and plugins. Contributing back to
the dfhack repository is welcome and the right thing to do :)</p>
<div class="section" id="df-data-structure-definitions">
<h2><a class="toc-backref" href="#id14">DF data structure definitions</a></h2>
<p>DFHack uses information about the game data structures, represented via xml files in the library/xml/ submodule.</p>
<p>Data structure layouts are described in files following the df.*.xml name pattern. This information is transformed by a perl script into C++ headers describing the structures, and associated metadata for the Lua wrapper. These headers and data are then compiled into the DFHack libraries, thus necessitating a compatibility break every time layouts change; in return it significantly boosts the efficiency and capabilities of DFHack code.</p>
<p>Global object addresses are stored in symbols.xml, which is copied to the dfhack release package and loaded as data at runtime.</p>
</div>
<div class="section" id="remote-access-interface">
<h2><a class="toc-backref" href="#id15">Remote access interface</a></h2>
<p>DFHack supports remote access by exchanging Google protobuf messages via a TCP socket. Both the core and plugins can define remotely accessible methods. The <tt class="docutils literal"><span class="pre">dfhack-run</span></tt> command uses this interface to invoke ordinary console commands.</p>
<p>Currently the supported set of requests is limited, because the developers don't know what exactly is most useful.</p>
<p>Protocol client implementations exist for Java and C#.</p>
</div>
<div class="section" id="contributing-to-dfhack">
<h2><a class="toc-backref" href="#id14">Contributing to DFHack</a></h2>
<h2><a class="toc-backref" href="#id16">Contributing to DFHack</a></h2>
<p>Several things should be kept in mind when contributing to DFHack.</p>
<div class="section" id="coding-style">
<h3><a class="toc-backref" href="#id15">Coding style</a></h3>
<h3><a class="toc-backref" href="#id17">Coding style</a></h3>
<p>DFhack uses ANSI formatting and four spaces as indentation. Line
endings are UNIX. The files use UTF-8 encoding. Code not following this
won't make me happy, because I'll have to fix it. There's a good chance
I'll make <em>you</em> fix it ;)</p>
</div>
<div class="section" id="how-to-get-new-code-into-dfhack">
<h3><a class="toc-backref" href="#id16">How to get new code into DFHack</a></h3>
<h3><a class="toc-backref" href="#id18">How to get new code into DFHack</a></h3>
<p>You can send patches or make a clone of the github repo and ask me on
the IRC channel to pull your code in. I'll review it and see if there
are any problems. I'll fix them if they are minor.</p>
Expand All @@ -506,7 +521,7 @@ <h3><a class="toc-backref" href="#id16">How to get new code into DFHack</a></h3>
fixing.</p>
</div>
<div class="section" id="memory-research">
<h3><a class="toc-backref" href="#id17">Memory research</a></h3>
<h3><a class="toc-backref" href="#id19">Memory research</a></h3>
<p>If you want to do memory research, you'll need some tools and some knowledge.
In general, you'll need a good memory viewer and optionally something
to look at machine code without getting crazy :)</p>
Expand Down
192 changes: 178 additions & 14 deletions LUA_API.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,26 @@ DFHack Lua API

.. contents::

====================
DF structure wrapper
====================
The current version of DFHack has extensive support for
the Lua scripting language, providing access to:

1. Raw data structures used by the game.
2. Many C++ functions for high-level access to these
structures, and interaction with dfhack itself.
3. Some functions exported by C++ plugins.

Lua code can be used both for writing scripts, which
are treated by DFHack command line prompt almost as
native C++ commands, and invoked by plugins written in c++.

This document describes native API available to Lua in detail.
For the most part it does not describe utility functions
implemented by Lua files located in hack/lua/...


=========================
DF data structure wrapper
=========================

DF structures described by the xml files in library/xml are exported
to lua code as a tree of objects and functions under the ``df`` global,
Expand Down Expand Up @@ -426,13 +443,17 @@ not destroy any objects allocated in this way, so the user
should be prepared to catch the error and do the necessary
cleanup.

================
DFHack utilities
================
==========
DFHack API
==========

DFHack utility functions are placed in the ``dfhack`` global tree.

Currently it defines the following features:
Native utilities
================

Input & Output
--------------

* ``dfhack.print(args...)``

Expand All @@ -451,6 +472,7 @@ Currently it defines the following features:
* ``dfhack.color([color])``

Sets the current output color. If color is *nil* or *-1*, resets to default.
Returns the previous color value.

* ``dfhack.is_interactive()``

Expand All @@ -473,15 +495,27 @@ Currently it defines the following features:

If the interactive console is not accessible, returns *nil, error*.


Exception handling
------------------

* ``dfhack.error(msg[,level[,verbose]])``

Throws a dfhack exception object with location and stack trace.
The verbose parameter controls whether the trace is printed by default.

* ``qerror(msg[,level])``

Calls ``dfhack.error()`` with ``verbose`` being *false*. Intended to
be used for user-caused errors in scripts, where stack traces are not
desirable.

* ``dfhack.pcall(f[,args...])``

Invokes f via xpcall, using an error function that attaches
a stack trace to the error. The same function is used by SafeCall
in C++, and dfhack.safecall.

The returned error is a table with separate ``message`` and
``stacktrace`` string fields; it implements ``__tostring``.

* ``safecall(f[,args...])``, ``dfhack.safecall(f[,args...])``

Just like pcall, but also prints the error using printerr before
Expand All @@ -491,6 +525,34 @@ Currently it defines the following features:

Compares to coroutine.resume like dfhack.safecall vs pcall.

* ``dfhack.exception``

Metatable of error objects used by dfhack. The objects have the
following properties:

``err.where``
The location prefix string, or *nil*.
``err.message``
The base message string.
``err.stacktrace``
The stack trace string, or *nil*.
``err.cause``
A different exception object, or *nil*.
``err.thread``
The coroutine that has thrown the exception.
``err.verbose``
Boolean, or *nil*; specifies if where and stacktrace should be printed.
``tostring(err)``, or ``err:tostring([verbose])``
Converts the exception to string.

* ``dfhack.exception.verbose``

The default value of the ``verbose`` argument of ``err:tostring()``.


Locking and finalization
------------------------

* ``dfhack.with_suspend(f[,args...])``

Calls ``f`` with arguments after grabbing the DF core suspend lock.
Expand Down Expand Up @@ -529,7 +591,7 @@ Currently it defines the following features:


Persistent configuration storage
================================
--------------------------------

This api is intended for storing configuration options in the world itself.
It probably should be restricted to data that is world-dependent.
Expand Down Expand Up @@ -571,7 +633,7 @@ functions can just copy values in memory without doing any actual I/O.
However, currently every entry has a 180+-byte dead-weight overhead.

Material info lookup
====================
--------------------

A material info record has fields:

Expand Down Expand Up @@ -1148,6 +1210,11 @@ Internal API
These functions are intended for the use by dfhack developers,
and are only documented here for completeness:

* ``dfhack.internal.scripts``

The table used by ``dfhack.run_script()`` to give every script its own
global environment, persistent between calls to the script.

* ``dfhack.internal.getAddress(name)``

Returns the global address ``name``, or *nil*.
Expand All @@ -1156,14 +1223,38 @@ and are only documented here for completeness:

Sets the global address ``name``. Returns the value of ``getAddress`` before the change.

* ``dfhack.internal.getBase()``
* ``dfhack.internal.getVTable(name)``

Returns the pre-extracted vtable address ``name``, or *nil*.

Returns the base address of the process.
* ``dfhack.internal.getRebaseDelta()``

Returns the ASLR rebase offset of the DF executable.

* ``dfhack.internal.getMemRanges()``

Returns a sequence of tables describing virtual memory ranges of the process.

* ``dfhack.internal.memmove(dest,src,count)``

Wraps the standard memmove function. Accepts both numbers and refs as pointers.

* ``dfhack.internal.memcmp(ptr1,ptr2,count)``

Wraps the standard memcmp function.

* ``dfhack.internal.memscan(haystack,count,step,needle,nsize)``

Searches for ``needle`` of ``nsize`` bytes in ``haystack``,
using ``count`` steps of ``step`` bytes.
Returns: *step_idx, sum_idx, found_ptr*, or *nil* if not found.

* ``dfhack.internal.diffscan(old_data, new_data, start_idx, end_idx, eltsize[, oldval, newval, delta])``

Searches for differences between buffers at ptr1 and ptr2, as integers of size eltsize.
The oldval, newval or delta arguments may be used to specify additional constraints.
Returns: *found_index*, or *nil* if end reached.


Core interpreter context
========================
Expand Down Expand Up @@ -1231,6 +1322,42 @@ Features:
Invokes all listeners contained in the event in an arbitrary
order using ``dfhack.safecall``.


=======
Modules
=======

DFHack sets up the lua interpreter so that the built-in ``require``
function can be used to load shared lua code from hack/lua/.
The ``dfhack`` namespace reference itself may be obtained via
``require('dfhack')``, although it is initially created as a
global by C++ bootstrap code.

The following functions are provided:

* ``mkmodule(name)``

Creates an environment table for the module. Intended to be used as::

local _ENV = mkmodule('foo')
...
return _ENV

If called the second time, returns the same table; thus providing reload support.

* ``reload(name)``

Reloads a previously ``require``-d module *"name"* from the file.
Intended as a help for module development.

* ``dfhack.BASE_G``

This variable contains the root global environment table, which is
used as a base for all module and script environments. Its contents
should be kept limited to the standard Lua library and API described
in this document.


=======
Plugins
=======
Expand Down Expand Up @@ -1292,3 +1419,40 @@ sort

Does not export any native functions as of now. Instead, it
calls lua code to perform the actual ordering of list items.


=======
Scripts
=======

Any files with the .lua extension placed into hack/scripts/*
are automatically used by the DFHack core as commands. The
matching command name consists of the name of the file sans
the extension.

**NOTE:** Scripts placed in subdirectories still can be accessed, but
do not clutter the ``ls`` command list; thus it is preferred
for obscure developer-oriented scripts and scripts used by tools.
When calling such scripts, always use '/' as the separator for
directories, e.g. ``devel/lua-example``.

Scripts are re-read from disk every time they are used
(this may be changed later to check the file change time); however
the global variable values persist in memory between calls.
Every script gets its own separate environment for global
variables.

Arguments are passed in to the scripts via the **...** built-in
quasi-variable; when the script is called by the DFHack core,
they are all guaranteed to be non-nil strings.

DFHack core invokes the scripts in the *core context* (see above);
however it is possible to call them from any lua code (including
from other scripts) in any context, via the same function the core uses:

* ``dfhack.run_script(name[,args...])``

Run a lua script in hack/scripts/, as if it was started from dfhack command-line.
The ``name`` argument should be the name stem, as would be used on the command line.

Note that this function lets errors propagate to the caller.
Loading

0 comments on commit 59fb4da

Please sign in to comment.