From 37e69db0b0e0022f2f5d2fc0ba0b47a269a0dd52 Mon Sep 17 00:00:00 2001 From: bcwalrus Date: Mon, 28 Jun 2010 20:13:53 -0700 Subject: [PATCH] Moved README to reSt. Fixed .gitnore. --- .gitignore | 3 + README | 144 ------------------------------------------ README.rst | 179 +++++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 182 insertions(+), 144 deletions(-) delete mode 100644 README create mode 100644 README.rst diff --git a/.gitignore b/.gitignore index b5eb6f4ae88..2acb1645d97 100644 --- a/.gitignore +++ b/.gitignore @@ -29,6 +29,9 @@ TAGS *.egg-info/ dist/ +# Include top level distribution dir +!/dist/ + # Crepo fetched them ext/thirdparty/js/art-widgets/ ext/thirdparty/js/mootools-table/ diff --git a/README b/README deleted file mode 100644 index b11ac094ee4..00000000000 --- a/README +++ /dev/null @@ -1,144 +0,0 @@ -Welcome to the repository for Hue. - -********************************************************************* -This is the development-oriented readme. If you want to write notes for -end users, put them in dist/README! -********************************************************************* - -File Layout -=========== -The "core" stuff is in desktop/core/, whereas installable apps live in apps/. -Please place third-party dependencies in the app's ext-py/ directory. - -The typical directory structure for inside a component includes: - src/, for Python code - static/, for static HTML and js resources - templates/, for data to be put through a template engine - docs/, for helpful notes - -The python code is structured simply: - module/package.py -Where module may be "filebrowser" or "jobsub". Because it is unlikely that -there are going to be huge conflicts, we're going without a deep nested -hierarchy. - -URL Layout -========== -core/src/desktop/urls.py contains the current layout for top-level URLs. -For the URLs within your application, you should make your own urls.py which will -be automatically rooted at /yourappname/ in the global namespace. See -apps/hello/src/hello/urls.py for an example. - -Development Prerequisites -=========================== -1) On your host system, you need to have the python "virtualenv" package installed. - -2) Also, you'll need some library development packages installed on your system: - -Debian: - gcc - libldap2-dev - libmysqlclient-dev - libsasl2-dev - libsqlite3-dev - libssl-dev - libxml2-dev - libxslt-dev - python-dev - python-setuptools - -CentOS: - cyrus-sasl-devel - gcc - libxml2-devel - libxslt-devel - mysql - mysql-devel - openldap-devel - openssl - python-devel - python-setuptools - sqlite-devel - python-simplejson (for the crepo tool) - -MacOS (mac port): - liblxml - libxml2 - libxslt - mysql5-devel - simplejson (easy_install) - sqlite3 - -3) You need to have crepo installed, and preferably on your path. If it is not -on your path, set the environment variable CREPO to point to crepo.py from that -distribution. You can clone crepo from git://github.com/cloudera/crepo.git somewhere -else on your system. - - -Getting Started -=============== -1) git clone http://github.com/cloudera/hue.git ; cd hue - [ you've assumedly already done this, since you're reading this file! ] -2) make apps -3) Start the server - build/env/bin/desktop runserver_plus - - -Setting up Hadoop -===================== - -In order to start up a pseudodistributed cluster with the plugins enabled, -run ./tools/scripts/configure-hadoop.sh all - -After doing so, running "jps" should show all the daemons running (NN, JT, TT, DN) -and you should be able to see the web UI on http://localhost:50030/ and -http://localhost:50070/ - - -FAQ -=== - -Q: What does "Exception: no app!" mean? -A: Your template has an error in it. Check for messages from -the server that look like: - INFO:root:Processing exception: Unclosed tag 'if'. Looking for one of: else, endif - -Q. What do I do if I get "There was an error launching ..."? -A. Turn on debugging by issuing "dbug.cookie()" in a Firebug console. - - -Django Conventions -================== -If you need to name your urls -(http://docs.djangoproject.com/en/dev/topics/http/urls/#naming-url-patterns) -because there's ambiguity in the view, be sure to prefix the name -with the application name. The url name namespace is global. So "jobsub.list" is fine, -but "list" is not. - -We recently moved to Django 1.1, which supports the notion of URL namespaces: - http://docs.djangoproject.com/en/dev/topics/http/urls/#url-namespaces -We have yet to move over our URLs to this construct. Brownie points for the developer -who takes this on. (TODO) - -Using and Installing Thrift -=========================== -Thrift's runtime libraries are checked into thirdparty/, -as appropriate. Right now, we check in the generated code. -To generate the code, you'll need the thrift binary. -We keep a private copy of the Thrift tree, because -Thrift tends to be mired in a version-less morass. -Compile it like so: - -1) git clone http://github.com/dreiss/thrift.git -2) cd thrift -4) ./bootstrap.sh -5) ./configure --with-py=no --with-java=no --with-perl=no --prefix=$HOME/pub - We exclude python, java, and perl because they don't like - to install in prefix. If you look around at configure's --help, - there are environment variables that determine where those - runtime bindings are installed. -6) make && make install - -When preparing .thrift files, you can use she-bangs to generate -the python bindings like so: - #!/usr/bin/env thrift -r --gen py:new_style -o ../../../ diff --git a/README.rst b/README.rst new file mode 100644 index 00000000000..18ad881482e --- /dev/null +++ b/README.rst @@ -0,0 +1,179 @@ +Welcome to the repository for Hue +================================= + +.. note:: + This is the development-oriented readme. If you want to write notes for + end users, please put them in ``dist/README``. + +Hue is both a web UI for Hadoop and a framework to create interactive web +applications. It features a FileBrowser for accessing HDFS, JobSub and +JobBrowser applications for submitting and viewing MapReduce jobs, a Beeswax +application for interacting with Hive. On top of that, the web frontend +is mostly built from declarative widgets that require no JavaScript and are +easy to learn. + + +File Layout +=========== +The "core" stuff is in ``desktop/core/``, whereas installable apps live in +``apps/``. Please place third-party dependencies in the app's ext-py/ +directory. + +The typical directory structure for inside an application includes: + + src/ + for Python code + + conf/ + for configuration (``.ini``) files to be installed + + static/ + for static HTML and js resources + + templates/ + for data to be put through a template engine + + docs/ + for helpful notes + +The python code is structured simply as +``module/package.py``, +where module may be "filebrowser" or "jobsub". Because it is unlikely that +there are going to be huge conflicts, we're going without a deep nested +hierarchy. + + +URL Layout +========== +``core/src/desktop/urls.py`` contains the current layout for top-level URLs. +For the URLs within your application, you should make your own ``urls.py`` +which will be automatically rooted at ``/yourappname/`` in the global +namespace. See ``apps/hello/src/hello/urls.py`` for an example. + + +Development Prerequisites +=========================== +1. On your host system, you need to have the python "virtualenv" package + installed. + +2. Also, you'll need these library development packages installed on your + system: + + Debian: + * gcc + * libldap2-dev + * libmysqlclient-dev + * libsasl2-dev + * libsqlite3-dev + * libssl-dev + * libxml2-dev + * libxslt-dev + * python-dev + * python-setuptools + + CentOS: + * cyrus-sasl-devel + * gcc + * libxml2-devel + * libxslt-devel + * mysql + * mysql-devel + * openldap-devel + * openssl + * python-devel + * python-setuptools + * sqlite-devel + * python-simplejson (for the crepo tool) + + MacOS (mac port): + * liblxml + * libxml2 + * libxslt + * mysql5-devel + * simplejson (easy_install) + * sqlite3 + +3. You need to have crepo installed, and preferably on your path. If it is not + on your path, set the environment variable ``CREPO`` to point to ``crepo.py`` + from that distribution. You can clone crepo from + http://github.com/cloudera/crepo.git somewhere else on your system. + + +Getting Started +=============== +To build and get the core server running (without any helper daemons):: + + $ export HADOOP_HOME= + $ git clone http://github.com/cloudera/hue.git + $ cd hue + $ make apps + $ build/env/bin/desktop runserver_plus + +Now Hue should be running on http://localhost:8000. + + +Setting up Hadoop +================= +In order to start up a pseudodistributed cluster with the plugins enabled, +run:: + + $ ./tools/scripts/configure-hadoop.sh all + +After doing so, running ``jps`` should show all the daemons running (NN, JT, +TT, DN) and you should be able to see the web UI on http://localhost:50030/ and +http://localhost:50070/. + + +FAQ +=== +1: What does "Exception: no app!" mean? + Your template has an error in it. Check for messages from the server that + look like:: + + INFO:root:Processing exception: Unclosed tag 'if'. Looking for one of: else, endif + +2: What do I do if I get "There was an error launching ..."? + Turn on debugging by issuing ``dbug.cookie()`` in a Firebug console. + + +Django Conventions +================== +If you need to name your urls +(http://docs.djangoproject.com/en/dev/topics/http/urls/#naming-url-patterns) +because there's ambiguity in the view, be sure to prefix the name +with the application name. The url name namespace is global. So +``jobsub.list`` is fine, but ``list`` is not. + +Hue is using Django 1.1, which supports the notion of URL namespaces: +http://docs.djangoproject.com/en/dev/topics/http/urls/#url-namespaces. +We have yet to move over our URLs to this construct. Brownie points for the +developer who takes this on. + + +Using and Installing Thrift +=========================== +Right now, we check in the generated thrift code. +To generate the code, you'll need the thrift binary. +Compile it like so:: + + $ git clone http://github.com/dreiss/thrift.git + $ cd thrift + $ ./bootstrap.sh + $ ./configure --with-py=no --with-java=no --with-perl=no --prefix=$HOME/pub + +We exclude python, java, and perl because they don't like +to install in prefix. If you look around at configure's --help, +there are environment variables that determine where those +runtime bindings are installed. +:: + + $ make && make install + +When preparing ``.thrift`` files, you can use she-bangs to generate +the python bindings like so:: + + #!/usr/bin/env thrift -r --gen py:new_style -o ../../../ + +.. note:: + This file is in reStructuredText. You may run + ``rst2html README.rst > README.html`` to produce a HTML.