More flexible dimension specification by the user

[sjperkins]

Currently montblanc wants things as a cube. This isn't so flexible going forward so I've been mulling over ideas for this.

Currently montblanc users inform montblanc about the size of their dimensions in a SourceProvider:

class MrSourceProvider(SourceProvider):
  """ Tell montblanc we're handling 64 channels """
  def updated_dimensions(self):
    return [('nchan', 64), ('ntime', 100)]

In the band case, we want to tell Montblanc that it should handle channels in separate groups...

class MrSourceProvider(SourceProvider):
  """
  Tell montblanc there's a group of size 4 with
  64, 32, 64 and 32 channels in the following way
  """
  def updated_dimensions(self):
    return [('nchan', 64, 32, 64, 32), ('ntime', 100)]

We can also introduce the idea of linking a dimension with a dimension or group of dimensions

class MrSourceProvider(SourceProvider):
  """
  Tell montblanc that nbands goups
  64, 32, 64 and 32 channels respectively.
  band 0: 64 channels
  band 1: 32 channels
  band 2: 64 channels
  band 3: 32 channels
  """
  def updated_dimensions(self):
    return [{'nband' : ('nchan', 64, 32, 64, 32)}, ('ntime', 100)]

Also if the users wants something really exotic, then the user could supply a callable/lambda that would allow them to configure the number of channels whenever that dimension is accessed in a loop. So user could then write logic like

• If this is a timestep 2 and baseline 3, we have 32 channels

def cfg_nchan(context):
  if context.ntime == 2 and context.nbl == 3:
    return 32
  else:
    return 64

class MrSourceProvider(SourceProvider):
  """ Configure nchan with cfg_nchan whenever we loop with it...  """
  def updated_dimensions(self):
    return [('nchan': cfg_nchan)]

[sjperkins]

@jtlz2 mentioned UVFITS/random groups - https://archive.stsci.edu/fits/users_guide/node28.html

[o-smirnov]

The latter option will be required to support baseline-dependent averaging. In the extreme case, each baseline will have its own individual time axis, and each timeslot of that baseline can have its own subset of channels. Is this too flexible to implement efficiently?

[sjperkins]

The latter option will be required to support baseline-dependent averaging. In the extreme case, each baseline will have its own individual time axis, and each timeslot of that baseline can have its own subset of channels. Is this too flexible to implement efficiently?

It should be fine. Only thing that I'd ask the user via API is a size hint for the maximum size for (GPU) memory budgeting purposes.