You might also want to look into autoencoder-based quantization or neural compression techniques, especially if you need extremely low-bandwidth representations for deployment.

For instance:

• Vector Quantized-VAEs (VQ-VAE) can produce discrete compressed representations that are both compact and semantically meaningful.

• Works really well when you need compact codes and don’t mind some loss in reconstruction fidelity as long as the classification performance holds.

Additionally, if the signals have a temporal structure (like ECG, sensor data, etc.), you can explore:

• Temporal Convolutional Networks (TCNs) as encoders instead of RNNs (often more efficient).

• Or even transformers with low-rank attention or sparse variants if your sequence lengths are large but you want to retain contextual information.

Finally, for streaming scenarios — I’d also suggest looking into online contrastive learning methods like Online BYOL or using continual learning frameworks to avoid forgetting while updating the encoder.

A good resource to explore:

• “Neural Data Compression” (Google Research blog / arXiv) – gives a solid grounding in modern approaches that mix compression with task-awareness.

Let me know if anyone has benchmarked VIB vs VQ-VAE in streaming setups—would love to hear comparisons!