Suggestions regarding default parameters in the LeHDC model implemented by TorchHD

[yby0313]

In the paper "LeHDC: Learning-Based Hyperdimensional Computing Classifier," the authors provide the following default parameters for the MNIST image recognition task: lr = 0.01, weight_decay = 0.05, dropout_rate = 0.5. Using these parameters, the authors achieved a 94.74% recognition rate for MNIST at a dimension of 10000.

However, training the LeHDC model provided by TorchHD with the same parameters for the MNIST image recognition task fails to achieve comparable performance.

In my analysis, this difference may be due to the fact that the LeHDC model implemented by TorchHD uses Base-Level encoding, while the encoding model in the aforementioned paper uses RP method.

This difference can be improved by modifying the parameters. Specifically, setting "lr = 0.01, weight_decay = 0.05, dropout_rate = 0.025" can achieve comparable recognition accuracy in the TorchHD-provided LeHDC model.

In conclusion, modifying the default parameters of the LeHDC model provided by TorchHD according to the above content and adding explanations of the differences in the encoding scheme may be helpful for readers to debug.

[mikeheddes]

Thank you for opening this issue. It would be great if the implementation in Torchhd matches the one used in the LeHDC paper. Can you provide more information (code samples would be great) on what you tried and how you fixed it?

You mentioned that the Torchhd implementation uses a different encoding function, can you elaborate on this? As per the paper: "we adopt the commonly-used record-based encoding" where the values are encoded using level hypervectors as described here:

This matches the encoding in Torchhd:

def encoder(self, samples: Tensor) -> Tensor:
        return functional.hash_table(self.keys.weight, self.levels(samples)).sign()

[yby0313]

Thank you very much for your careful response. Regarding the issue of the encoding method, it was indeed my oversight. Both of them use the same Base-Level approach.

However, when the LeHDC model built based on TorchHD was used for the MNIST image recognition task with the parameters recommended in the paper (dimension = 10000, lr = 0.01, weight_decay = 0.05, dropout_rate = 0.5, batch_size=64), the average recognition accuracy of the three training and inference runs was 87.73% (87.70%, 87.77%, 87.71% ).

If dropout_rate was changed to 0.025, the average recognition accuracy of the three runs was 94.37% (94.09%, 94.58%,94.44% ).

Therefore, there might be other different factors here that have led to the difference in recognition performance.