|
| 1 | +Silo -- a simple, general purpose file system for LSL via HTTP |
| 2 | + version 2006-07-09-beta |
| 3 | + by Zero Linden |
| 4 | + |
| 5 | + Copyright (c) 2006 Linden Lab |
| 6 | + Licensed under the "MIT" open source license. |
| 7 | + |
| 8 | + |
| 9 | +==== ABOUT ==== |
| 10 | + |
| 11 | +Silo stores and retrieves data in an (almost) arbitrary tree of |
| 12 | +URLs on a web server. It is very similar to a file system. It was |
| 13 | +written to provide data storage for LSL scripts in Second Life. |
| 14 | +However, it is general enough to be used from other languages and |
| 15 | +systems, and to even store other kinds of data. (Though LSL can |
| 16 | +only access text.) |
| 17 | + |
| 18 | + |
| 19 | +==== INSTALLATION ==== |
| 20 | + |
| 21 | +Silo is installed on a web server that runs PHP. You need PHP |
| 22 | +version 4.3.0 or later. The instructions here are for the Apache |
| 23 | +web server and work with version 1 or 2 of the server. |
| 24 | + |
| 25 | +The installed silo script will have a "base URL" that is the start |
| 26 | +of the storage tree. It will be highly dependent on your server |
| 27 | +configuration and the method you choose for installation. This |
| 28 | +base URL is the access point to your silo. |
| 29 | + |
| 30 | +1) Pick a directory that can be served by the Apache web server. |
| 31 | +You might want to make a subdirectory just for silo. |
| 32 | + |
| 33 | +2) Put the file silo.php in that directory. |
| 34 | + |
| 35 | +3) Create a directory called data, and make sure that its permissions |
| 36 | +are set so that the Apache server can write to it. It's not |
| 37 | +unreasonable to simply make it writable by all. After all, that |
| 38 | +is what you are doing by deciding to store data via HTTP anyway! |
| 39 | + |
| 40 | +4) Depending on how your Apache server is set up, you may need |
| 41 | +changes to the Apache configuration. Here are some examples of |
| 42 | +common Apache set-ups, but more complicated ones are certainly |
| 43 | +possible. |
| 44 | + |
| 45 | + |
| 46 | + Configuration 1: Under the document root |
| 47 | + ---------------------------------------- |
| 48 | + Apache's config file contains: |
| 49 | + DocumentRoot /var/www/htdocs |
| 50 | + |
| 51 | + You create the directory: |
| 52 | + /var/www/htdocs/sl-stuff |
| 53 | + |
| 54 | + Place silo.php there, and create data there with full permissions. |
| 55 | + |
| 56 | + The base URL for the silo will be: |
| 57 | + http://www.example.com/sl-stuff/silo.php |
| 58 | + |
| 59 | + |
| 60 | + Configuration 2: In user's public-html dir |
| 61 | + ------------------------------------------ |
| 62 | + Apache's config file contains: |
| 63 | + UserDir public_html |
| 64 | + |
| 65 | + You create the directory: |
| 66 | + ~you/public_html/sl-stuff |
| 67 | + |
| 68 | + Place silo.php there, and create data there with full permissions. |
| 69 | + |
| 70 | + The base URL for the silo will be: |
| 71 | + http://www.example.com/~you/sl-stuff/silo.php |
| 72 | + |
| 73 | + |
| 74 | + Configuration 3: Outside the document tree |
| 75 | + ------------------------------------------ |
| 76 | + Apache's config file contains: |
| 77 | + Alias /silo/ /var/sl-stuff/silo/silo.php/ |
| 78 | + <Directory /var/sl-stuff/silo> |
| 79 | + Order allow,deny |
| 80 | + Allow from all |
| 81 | + </Directory> |
| 82 | + |
| 83 | + You create the directory: |
| 84 | + /var/sl-stuff/silo |
| 85 | + |
| 86 | + Place silo.php there, and create data there with full permissions. |
| 87 | + |
| 88 | + The base URL for the silo will be: |
| 89 | + http://www.example.com/silo |
| 90 | + |
| 91 | + Note that in this configuration, the directory needn't be under |
| 92 | + the document root, or the mapped user directories. It is the |
| 93 | + Alias directive that maps the directory, and provides the nice, |
| 94 | + clean base URL. |
| 95 | + |
| 96 | +5) Test the set up by copying the files test.py and uuid.py to any |
| 97 | +machine that has python installed. It needn't be the same machine |
| 98 | +as the server. In a command shell, change to the the directory |
| 99 | +with test.py and uuid.py, and run: |
| 100 | + |
| 101 | + python test.py <baseURL> -v |
| 102 | + |
| 103 | +Replacing <baseURL> with your actual base URL. Note: This URL does |
| 104 | +NOT end in a slash. |
| 105 | + |
| 106 | +This should report that it passes all the tests. If not, something |
| 107 | +might be wrong with your configuration, or there may be some |
| 108 | +incompatibility between the script and your system. If you suspect |
| 109 | +the later, let me know the details! |
| 110 | + |
| 111 | + |
| 112 | +==== CONCEPTS ==== |
| 113 | + |
| 114 | +Data is stored at paths under the base URL. There are some |
| 115 | +restrictions on the path: |
| 116 | + - the path components can contain only characters in the set: |
| 117 | + - + _ % 0-9 a-z A-Z |
| 118 | + Note that URL encoding any string meets these requirements, |
| 119 | + as does calling LSL's llEscapeURL function. |
| 120 | + - there must be at least one path component |
| 121 | + - there can be no more than 11 path components |
| 122 | + |
| 123 | +For example, any of these are valid paths: |
| 124 | + /tuna+fish |
| 125 | + /007 |
| 126 | + /fruit/apple |
| 127 | + /fruit/apple/fuji |
| 128 | + /fruit/apple/gala |
| 129 | + /fruit/banana |
| 130 | + /9c84d7e2-713f-4269-a27b-14b133a0ec56 |
| 131 | + |
| 132 | +Notice that unlike most file systems, you can store data at both a |
| 133 | +path ("/apple"), and at paths below it ("/apple/fuji"). |
| 134 | + |
| 135 | +Using a UUID (or key in LSL) gives silo some optionally strong |
| 136 | +security. Since keys are unguessable, when store a tree of data |
| 137 | +under a path starting with a key, there is no way for anyone to |
| 138 | +access that data, unless you give them the key. |
| 139 | + |
| 140 | + |
| 141 | +==== USE ==== |
| 142 | + |
| 143 | +Data is stored and accessed via standard HTTP methods with a path: |
| 144 | + GET - fetch data at the path |
| 145 | + PUT - store data at the path |
| 146 | + DELETE - remove data at the path |
| 147 | + |
| 148 | +When you end a path in a slash, these methods can be used: |
| 149 | + GET - fetch a list of path parts under this one, one per line |
| 150 | + DELETE - delete all paths under this one |
| 151 | + |
| 152 | +When storing data (PUT), you must be sure that both the 'content-type' |
| 153 | +(MIME Type) and 'content-length' headers are set. When accessing |
| 154 | +from LSL, these are automatically set. |
| 155 | + |
| 156 | +All operations will return a non-error status (2xx) if the operation |
| 157 | +completed correctly. For the PUT operation, you can use the status |
| 158 | +code to discover if the path was newsly created: It returns |
| 159 | +201 in that case. |
| 160 | + |
| 161 | +There is no need to store data a intermediate nodes before writing |
| 162 | +something lower down. If you are storing at "/apple/fuji", you |
| 163 | +needn't have stored anything at "/apple". |
| 164 | + |
| 165 | +Note that you cannot use a path of just "/". This means that no |
| 166 | +one can delete the entire silo, nor can anyone find out all the |
| 167 | +paths in the silo. Because of this, if you use an unguessable UUID |
| 168 | +as the first path component, other users of the same silo cannot |
| 169 | +access your data unless you give them the UUID. |
| 170 | + |
| 171 | + |
| 172 | +---- From LSL ---- |
| 173 | + |
| 174 | +You can make access easier by setting up this global: |
| 175 | + string gBase = "http://www.example.com/silo"; |
| 176 | + |
| 177 | +Storing data: |
| 178 | + string data = "something to store"; |
| 179 | + llHTTPRequest(gBase + "/apple", [ HTTP_METHOD, "PUT" ], data); |
| 180 | + |
| 181 | +Fetching data: |
| 182 | + llHTTPRequest(gBase + "/apple", [ ], ""); |
| 183 | + ... |
| 184 | + http_response(key req, integer status, list meta, string content) |
| 185 | + { |
| 186 | + if (status != 200) { |
| 187 | + llOwnerSay("there was a problem: status = " + (string)status); |
| 188 | + } |
| 189 | + else { |
| 190 | + data = content; |
| 191 | + } |
| 192 | + } |
| 193 | + |
| 194 | +Deleting data: |
| 195 | + llHTTPRequest(gBase + "/apple", [ HTTP_METHOD, "DELETE" ], ""); |
| 196 | + |
| 197 | + |
| 198 | +If you want to use the method of storing your data under a key, |
| 199 | +then you can set things up like this: |
| 200 | + string gSilo = "http://www.example.com/silo/"; |
| 201 | + string gBase; |
| 202 | + |
| 203 | + initBase() { |
| 204 | + string aKey; |
| 205 | + |
| 206 | + aKey = (string)llGetInventoryKey("storage key"); |
| 207 | + // make a notecard and add it to the inventory of each |
| 208 | + // object that is accessing this data. be sure the owner of |
| 209 | + // the objects have modify, copy and transfer permissions |
| 210 | + |
| 211 | + // or |
| 212 | + aKey = (string)llGetKey(); |
| 213 | + // only if this is not the root prim (otherwise the object key |
| 214 | + // can be scanned for, be sure also that this prim doesn't talk |
| 215 | + |
| 216 | + // or |
| 217 | + aKey = "9c84d7e2-713f-4269-a27b-14b133a0ec56" |
| 218 | + |
| 219 | + // but never |
| 220 | + aKey = (string)llGetOwner(); |
| 221 | + // as the avatar keys are easily obtained |
| 222 | + |
| 223 | + gBase = gSilo + "/" + aKey; |
| 224 | + } |
| 225 | + |
| 226 | +---- From CURL ---- |
| 227 | + |
| 228 | +If you have access to 'curl' from a command shell, you can try out the |
| 229 | +silo easily using curl: |
| 230 | + |
| 231 | + SILO=http://www.example.com/silo |
| 232 | + |
| 233 | + # Storing data |
| 234 | + echo "some data" | curl --data-binary @- -X PUT -H 'Content-Type: text/plain' $SILO/apple |
| 235 | + |
| 236 | + # Fetching data |
| 237 | + curl $SILO/apple |
| 238 | + |
| 239 | + # Deleting data |
| 240 | + curl -X DELETE $SILO/apple |
| 241 | + |
| 242 | + |
| 243 | +==== CONTACT ==== |
| 244 | + |
| 245 | +This script was written by Zero Linden. You can conact him at |
| 246 | + |
| 247 | + |
| 248 | + |
0 commit comments