Portable types (Vector, Matrix, Quaternion, Vec2D, MapGrid2D, Image, Sound, PointCloud)

[drdanz]

I have a huge confusion in my head about these classes, hence I'm opening this issue to discuss about this. I think we should start clarifying a few things. I'll just leave a few discussion points and a few ideas here, just to start...

---

I see 3 types of objects here:

• Container objects (should be in OS?)
  • ManagedBytes
  • Vector (maybe?)
• Mathematical objects (Should be in math, but math is not built by default)
  • Vector
  • Matrix
  • Quaternion
  • Vec2D (what is the difference with Vector?)
• Media objects (Should be in sig?)
  • Image
  • Sound
  • PointCloud
  • MapGrid2D (what is the difference with Image?)

---

I remember there was a discussion at some point, where it came out that Vector is serialized better than std::vector for some reason. Was it just in thrift, or was it somewhere else? If it is in thrift, we should definitely improve it. In yarpidl_rosmsg, vectors are stored using std::vector and serialized/de-serialized using connection.appendExternalBlock (in "bare" mode) or with a for loop and appendDouble() (in "text" mode). I think this makes sense.

---

Vector is used in several places as a container and in many places as mathematical vector. This is really confusing, I think we should have a Portable vector" that is just a storage, and that is used internally by the mathematical vector.

---

What do you think about having "portable containers" (vector, array, list, and more), possibly implemented as

template<class T, class Allocator = std::allocator<T>>
class vector : public std::vector<T, Allocator>
               public yarp::os::Portable
{
// Implement only Portable here
};

This will be some sort of compatibility layer between c++ and YARP, something similar to opencv and yarp images, and pcl and yarp point clouds.

[jgvictores]

Related: #1559

Statement: Fully reasoning on this comment, we'd say dev can use sig, math can use sig, and dev should not use math.
Thus, sig=media can be used as mathematical objects, which makes sense: store an image using sig, then do some mathematical operations upon it via math.

What would be left unresolved is where robotic classes (map, pose/quaternion) go. Two options here:

1. If they go to sig, all situations are resolved.
2. If they go to dev, we'd have to add a math can use dev axiom too.
3. If they go to new_robotics, we'd have to add a math can use new_robotics axiom too.

I left out options that contradict the first statement (putting them in math, or leaving them in some ROS-related code). I'm slightly inclined towards dev for semantic reasons, but towards sig for practical reasons.

Another topic is rather specific types should even exist (as you state, MapGrid2D vs Image). IMHO, types are good, even if it's just a typdef or raw public inheritance that adds no extra functionality.

---

n/a

---

This is really confusing, I think we should have a Portable vector" that is just a storage, and that is used internally by the mathematical vector.

Sounds good, see below.

---

possibly implemented as (...)

That implementation LGTM. :-)

[jgvictores]

Just to clarify, IMHO...

I understand the conception of small vector mathematical operations vs big media objects.
However, the trend is to want transmit big objects (= media sound/image, robotics pose/map, or thin-red-line pointclouds) efficiently (thus the portable container object below), and then operate directly upon these objects (thus setting the math layer above, maybe even look at yarp::sig::file/yarp::sig::draw for inspiration).

Following this reasoning, the general use case is that layer set up should look more like this:

• Low-level: Container objects (should be in OS? OK)
• Mid-level: Media objects (Should be in sig? see sig vs sig/dev/new_namespace(s) above)
• Top-level: Mathematical objectsoperations (Should be in math, but math is not built by default? as a top layer, this should be okay)

The additional advantage of this is maintaining a certain degree of modularity and avoid mutual dependencies, enabling at least being able to (1) freely disable the top math layer (2) freely disable the above sig/dev and math layers.

It would be great to know different opinions! :-)

[lornat75]

If I remember correctly the logic in the current separation was more or less:

Place in sig objects that could have been useful for handling signals, this is why Vector and Matrix (as a general purpose storages for analog signals, i.e. inertial, force, etc) ended up there.

math was intended to keep mathematical operations and algorithms (like random generation), with additional dependency on linear algebra libraries (initially GSL then eigen). It ended up containing other mathematical entities that did not fit in sig.

I agree that portable containers could go to os, but mathematical entities like Vector and Matrix should be in sig.

If you think that other "mathematical entities" (like Quaternion and Vec2D) should not be separated from Vector and Matrix then I would be inclined to move them to sig.

[drdanz]

It's not very clear the meaning of "signal" here...
If I get it correctly, types are in sig, but mathematical operations involving these types are not in the same library (i.e. they are in math), because you consider them "signals", but as I see it, a force is an entity that is represented by a "mathematical vector", it is not a signal. But I believe that with signal, you mean something that can be transmitted on the network, i.e. more or less anything...

I believe that the original reason for having them separate, was that the math library was dependent on a GPL library, therefore when you wrote the library you had to separate them, in order to be able to transmit them, without actually using them, even when the math library was not available. Am I wrong here?

This situation at the moment is very odd, and it implies that in order to do use the operators for yarp::sig::Vector, besides including yarp/sig/Vector.h and linking YARP_sig you have to include yarp/math/math.h, add a using namespace math, and link YARP_math. Even if we move them out of the namespace (see #1602) you still need use the namespace and link the library.

I know this is a big breaking change, but I think that the clean way to handle this would be to have a "storage vector" in os defining the serialization of the object, and a "mathematical vector" in math that derives from the "storage vector" adding mathematical operations and cast operator, so that they can be easily exchanged.

[lornat75]

It is true that operators that act on Vector and Matrix are in a separate library and that this is odd. Unfortunately some (if not most) operators with Vector and Matrix are implemented using linear algebra functions and therefore, as you say, have been separated (implementing our own routines would hardly be as efficient).

They can be used in isolation, or with another linear algebra library, in my opinion their existence in the YARP_sig library has some use, even without YARP_math. Having them in YARP_math on the other hand would force everyone that needs a Vector or a Matrix to use YARP_math and its dependency.

The implementation of storage vector in YARP_OS (which could be useful, as a thin wrapper on std::vector with YARP serialization/deserialization functions) would provide a close equivalent to Vector, but what about Matrix?

[pattacini]

Refactoring yarp::sig::Vector and yarp::sig::Matrix and the other types in order to make them rely as close as possible on C++11 containers - that in turn, I think, will need to be portable as well - might be a good idea, if we decide that this effort will pay off. In theory, it could be nice just writing a std::vector object (or equivalent containers) to the net.

A different thing is instead moving Vector/Matrix out of yarp::sig. I'm against this option and I don't see any problem in having to deal with yarp::math library for providing the corresponding math.

All the other signals (e.g. quaternions, sound, point-cloud) could have potentially their own math, even though we haven't defined it yet. To that end, one might need to resort to proper external libraries to implement those operations efficiently; that's why separating signals from their math (available, yet-to-come, won't-provide) is useful and wieldy.

[drdanz]

Again... some confusion here. There is no "matrix" data structure in computer science, a matrix can be implemented using different methods, a single array (possibly with padding between lines), an array of array; it can be stored by rows, or by columns.
The same kind of data structure is used for storing image, sound, mapgrid2d etc, therefore it make sense that the serialization of the object should be shared, and therefore handled by the underlying structure, not by the "Matrix" class itself.

Yes, in case of the Matrix class, the underlying structure does not have the methods to access elements by rows and columns... Perhaps we could have a yarp::os::PortableMatrix class (I'm starting to consider using PortableVector, PortableTuple, etc, since we already have a PortablePair), but I'm still trying to figure out how to handle the different underlying possible structures.