|
Riak Secondary Indexes Sample |
|
Written by Gordon Tillman
|
|
Sunday, 13 November 2011 19:51 |
Introduction
An interested question was posted on the the Riak Users mailing list. Here are the application requirements:
The basic overview is this:
50K devices push data once a second to web services which need to store that data in short-term storage (Riak).
Once an hour, a sweeper needs to take an hour's worth of data per device (if there is any) and ship it off to
long term storage, then delete it from short-term storage. Ideally, there'd only ever be slightly more than
1 hour's worth of data still in short-term storage for any given device. The goal is to write down the data
as simply and safely as possible, with little or no processing on that data.
Each second's worth of data is:
* A device identifier
* A timestamp (epoch seconds, integer) for the slice of time the data represents
* An opaque blob of binary data (2 to 4k)
Once an hour, I'd like to do something like:
* For each device:
* Find (and concat) all the data between time1 and time2 (an hour).
* Move that data to long-term storage (not Riak) as a single blob.
* Delete that data from Riak.
|
|
Last Updated on Sunday, 13 November 2011 20:14 |
|
Read more...
|
|
|
Written by Gordon Tillman
|
|
Monday, 04 May 2009 15:36 |
|
This is just a brief write-up on the steps that I took to get a basic Django application up and runnning with an On-Rev hosting account. To avoid having to embed a bunch of hyperlinks in the main text, I've include a complete references section at the bottom of the post. I'm going to assume that you have contacted On-Rev support and gotten them to grant you SSH access to your account. That is necessary to complete the following steps.
As far as I know, On-Rev does not currently support mod-python or FastCGI, so we are limited for now to using plain CGI. This is not very efficient but I think as the On-Rev service matures they will probably offer other options.
|
|
Read more...
|
|
Written by Gordon Tillman
|
|
Tuesday, 03 March 2009 17:40 |
|
I tend to update my fitness routines over time as my needs change and as I improve my understanding. One important goal with the most current revision of my workout schedule was to minimize the risk of injury. I had knee surgery last year and during the months that followed had to be very careful to not cause additional problems. What follows is based upon several references, which I shall list at the bottom of the article.
|
|
Last Updated on Tuesday, 03 March 2009 18:41 |
|
Read more...
|
|
|
Written by Gordon Tillman
|
|
Monday, 27 April 2009 17:52 |
|
Until now I've never had any firm standards on how I create and apply tags for images (think Flickr or other information. I did some searching and found precious little.
|
|
Last Updated on Monday, 27 April 2009 18:16 |
|
Read more...
|
|
Written by Gordon Tillman
|
|
Thursday, 22 January 2009 15:49 |
|
I ran into a minor problem recently when running a Python script with the cron facility. It was a simple oversight (and I know better) but I thought it may serve as a useful reminder.
Cron jobs run with a very small PATH. By default, it is /usr/bin:/bin. This is not normally a big deal, because you should be using full paths anyway in your scripts. But one place where that is not always followed is the shebang line at the top of a script. For example, the first line of a Python script may look like this:
#!/usr/bin/env python
Or, if you needed a particular version of Python:
#!/usr/bin/env python2.5
This is all well and good. But if, for example, python2.5 was located here: /usr/local/bin/python2.5... well that will not work in a cron job unless you override the default path. You will not receive any errors ... it just won't run. The fix in this case is just to put this at the top of your crontab file:
PATH=/usr/local/bin:/usr/bin:/bin
|
|
Last Updated on Thursday, 22 January 2009 16:00 |
|
|
|
|
|
|
Page 1 of 7 |
