Enable definition of multiple merge windows per node [JIRA: RIAK-2730]

[cdahlqvist]

In order to limit the potential impact of merging, it is often recommended to use non-overlapping, staggered merge windows. This currently means relying on a single merge window per node, leading to increased disk usage between windows due to the lack of merging activity.

This PR makes it possible to define a list of merge windows for a node, allowing the creation of multiple shorter merge windows for each node, potentially reducing the time between merges.

The list of merge windows can be specified as follows in the app.config file:

{merge_window, [{1, 1},{7, 7},{13, 13},{19, 19}]},

[joedevivo]

I've recently moved the bitcask cuttlefish schema into the bitcask repo in PR #112. After these are both merged, we'll need to revisit the merge_window section of the schema to make sure this can be set via cuttlefish.

[slfritchie]

The Riak 2.0 code freeze date is approaching. @cdahlqvist What work is required for cuttlefish-style configuration?

[joedevivo]

This PR needs to be rebased before that work can be done. A list of tuples is a weird thing to map. Here are some ideas of how it could be done:

start/end pairs

merge_window.1=1
merge_window.7=7
merge_window.13=19
merge_window.19=19

but I don't like that so much.

$named windows

merge_window.midnight.start = 1
merge_window.midnight.end = 1
merge_window.morning.start=7
merge_window.morning.end=7
merge_window.noon.start = 13
merge_window.noon.end = 13
merge_window.evening.start = 19
merge_window.evening.end = 19

We already have merge_window.start and merge_window.end, so this would fit in there. We could also just do:

Parsed list of tuples

merge_windows = [{1,1}, {7,7}, {13,13}, {19,19}]

but then I'll have to teach cuttlefish how to parse a string. not a terrible idea, but work that would need doing.

Do nothing

We could require that this be set in the advanced.config file.

[evanmcc]

i vote for the last one. not hard to walk people through.

[joedevivo]

Did this even get reviewed, or is the entire conversation just about configuration?

[evanmcc]

I reviewed but didn't test. Need to take another look and test it. But I
think that we're just talking about config, here.

On Fri, Dec 13, 2013 at 1:37 PM, Joe DeVivo [email protected]:

Did this even get reviewed, or is the entire conversation just about
configuration?

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/111#issuecomment-30545648
.

[joedevivo]

Don't let config slow this down. get it merged and assume advanced.config for now

[slfritchie]

Hrm. Please note that this PR, #127, has just been reviewed & merged. It isn't clear to me how much more work/change that this ticket requires, sorry, I don't have time this week to review the details.

[slfritchie]

Ping: Friday's deadline is Friday.

[evanmcc]

So, Jon called "no more features" before I could get this in. I am going to tweak it a little bit and add a test, then check it into the first version past the 2.0 tag for bitcask, so that it makes it into the next possible release past the initial 2.0 release.

I feel terrible, sorry Christian.

[angrycub]

create jira issue