Fixed-size string attributes include `\0` in resulting `std::string`

[FlyingSamson]

When I try to read a fixed-size string attribute with properties (from h5dump -e -a ...) like

ATTRIBUTE "Creator" {
   DATATYPE  H5T_STRING {
      STRSIZE 7;
      STRPAD H5T_STR_NULLTERM;
      CSET H5T_CSET_ASCII;
      CTYPE H5T_C_S1;
   }
   DATASPACE  SCALAR
   DATA {
   (0): "xxxxxx"
   }
}

using code like

std::string value;
attribute.read(value);

the resulting std::string value contains the terminating '\0' character as last character in its internal vector of characters.

I found the related issue #224, and am now unsure if this is intended behavior. I personally find this behavior rather odd and not really conforming to what I expected from an interface storing a string attribute into a std::string.

PS: Reading the same attribute but stored as non-fixed-size string in the hdf5 file

ATTRIBUTE "Creator" {
   DATATYPE  H5T_STRING {
      STRSIZE H5T_VARIABLE;
      STRPAD H5T_STR_NULLTERM;
      CSET H5T_CSET_ASCII;
      CTYPE H5T_C_S1;
   }
   DATASPACE  SCALAR
   DATA {
   (0): "xxxxxx"
   }
}

using the same c++ code, works as expected (i.e., no trailing \0 in the resulting std::string).

[jkotan]

Dear @FlyingSamson,

sorry for late replay. On the one hand I agree with you i.e. it would be nice to have c++ std::string without trailing \0. On the other currently we provide the exact string provided by c-like libhdf5. I would not like to have to much overhead which slow down reading e.g. by coping big string arrays form one place to another to erase \0. Do you have in mind any patch we could test and apply?

[FlyingSamson]

Hi @jkotan,

I see your point. I don't have a particular patch set in mind at the moment, but I would have hoped that one might find some solution based on the STRPAD attribute stored alongside of string variables in the HDF5 file. There seems to be H5Tget_strpad to retrieve this property.

In case of H5T_STR_NULLTERM I would say the intended behavior would be to read a string of length one less than indicated in the file.

H5T_STR_NULLPAD or H5T_STR_SPACEPAD are not quite as easy I think. I assume, ideally we would obtain strings with all trailing \0 or ' ' stripped respectively. But I don't see any way to know this in advance.
My best idea in this regard right now, is to have FixedLengthStringBuffer<std::string>::create() read the entire data from file into an internal buffer (just as it already does in the current implementation), and to update FixedLengthStringTrait<std::string>::from_buffer to not just copy the range buffer.begin() to buffer.end() using the range constructor of std::string but to first use std::find_if_not on the reverse iterators of buffer to search for the first element that is unequal to \0 or . And then use the range c'tor of string with buffer.begin() and the found end iterator.

I hope you roughly understand what I would intend to do? Would that be something that you would consider merging (in particular the part with the additional effort for the search I guess)?

In any case preparing a proper MR that implements and tests the above would definitely take me a while, so not something that should prevent you from making the release 0.7.0 (which is great by the way!).