Enable use of MLP as summary network

[vpratz]

The MLP class currently cannot be used as a summary network (AttributeError: 'MLP' object has no attribute 'compute_metrics').

I think it would be good to have MLPs available as general purpose summary networks, when none of the specialized summary networks is suitable for the problem at hand. A light wrapper around MLP would be sufficient.

Open questions:

1. Do we need a new class? My tendency is yes, because
   a) we might not want to incorporate SummaryNetwork arguments and functions in the general purpose MLP class
   b) the current implementation has an activation after the last layer, which is not desirable for a summary network, so the class should include an additional projector
2. Naming: How to avoid confusion with the MLP class. The most obvious name would be SummaryMLP.

@stefanradev93 @LarsKue What do you think, is this something we want to include?

[stefanradev93]

I agree, it may be a nice-to-have class if some level of dimensionality reduction is desirable for large vector-valued data (pretty rare use case tho) before concatenating it with target quantities directly in the inference networks. Perhaps call it DenseNetwork / FeedForwardNetwork to mirror TimeSeriesNetwork and be sufficiently different from MLP?

[LarsKue]

Doesn't DeepSet already fill this niche? It's about as basic as a Summary Network can get. What pooling would you apply with the MLP?

[vpratz]

DeepSet with set_size=1 would be similar, but this is not super obvious, and configuring it would be quite confusing due to its more complex architecture. The case I have in mind is just one observation with no known structure, so no pooling is necessary, and this would operate on already flattened data (or we flatten before the MLP, if we think this it is better to keep the additional dimension for the input).

[vpratz]

@LarsKue What interface/assumptions do you currently have in mind for the summary networks?
Should they all follow the signature (batch_size, set_size, feature_size) (or (batch_size, num_timesteps, feature_size) for time-series)? Or would we allow to drop the second dimension if we do not deal with exchangeable or time-series data?

[vpratz]

@LarsKue Bumping this. How is the strategy/plan for "base" networks, like MLP and ResNet, that encapsulate the complex part of an architecture, but require wrapping for actual use as a summary network? As we want to make it as easy for users as possible to use our networks, I think it would be good to provide a wrapped version as well, so that they do not have to know about writing their own classes to use those functionalities.
What do you think?

[LarsKue]

@vpratz I think the best solution would be if we provide a simple wrapper like a SimpleSummaryNetwork that can be used with plain feed-forward networks, and make pooling optional. I would not provide one version for every network, though. Simply let the user pass the feed-forward network in the constructor.