Streaming Continual Learning: MSc thesis on architectures

This is my MSc thesis in Computer Science and Engineering (Laurea Magistrale in Ingegneria Informatica) at Politecnico di Milano, with the working title Deep Learning Architectural Strategies for Streaming Continual Learning. It is about neural networks that learn from continuously changing data streams without forgetting what they have already learned. The work started in September 2026 and is at an early stage, so there are no results yet; graduation is expected in March 2027.

My supervisor is Prof. Emanuele Della Valle and my co-supervisor is Federico Giannini.

Context: what Streaming Continual Learning is

A lot of data arrives as a continuous flow, a stream: weather readings, air quality, power consumption. A model that learns from this kind of data has to handle four challenges together:

• continuous learning: it is updated as the data arrives, with no one-off offline training;
• concept drift: the data distribution changes over time, and sometimes the rule that maps inputs to labels changes too, even contradicting the previous one;
• catastrophic forgetting: while adapting to the new concept, the network risks erasing what it knew, knowledge that may be useful again if a concept comes back;
• temporal dependence: each data point depends on the ones before it.

Three research communities each cover part of the problem: Streaming Machine Learning adapts to drift but forgets, Continual Learning avoids forgetting but has mostly been studied with non-contradictory drifts, and time series analysis models temporal dependence but is rarely applied to streams. Streaming Continual Learning (SCL) brings these goals together: adapt quickly, keep past knowledge and reuse the part of it that matters for the current concept [1, 2].

Why architectural strategies

For data with temporal dependence, Giannini's PhD thesis proposes a recipe [1]: a recurrent network trained continuously on the stream (cRNN), which reads the last W points through a sliding window, plus a Continual Learning architectural strategy on top.

The reason is that, with contradictory drifts, replay- and regularisation-based strategies may fail to converge because they optimise all concepts jointly, whereas architectural strategies physically separate the parameters of different concepts [1, 2]. Architectural strategies, however, are usually applied to convolutional or feed-forward networks, and according to the same PhD thesis the literature lacks off-the-shelf ones that work continuously on recurrent networks [1].

Two families: columns and masks

In the reference works of the thesis, architectural strategies fall into two families.

Column-based. cPNN (Continuous Progressive Neural Networks) starts from a single cRNN, called a column [3]. At each drift it adds a column and freezes the previous ones; through lateral connections the new column also receives the previous column's hidden state alongside the input, so it reuses what has already been learned without overwriting it. This is the idea shown in the image at the top of the page. cPNN, however, needs an external drift detector and grows at every detection. DYNcPNN detects drifts on its own, decides when adding a column actually pays off and, when it does not add one, still protects the acquired knowledge from being forgotten [4].

Mask-based. Here the weights of a network stay frozen and are reused through learnable masks that modulate their contribution. Continuous Piggyback applies the masks to the weights of a frozen, pre-trained GRU [1]. GIN, introduced in Sandro D'Andrea's MSc thesis, combines weight masking, progressive network expansion and recurrent networks [5]. MAGIC Net applies learnable masks over frozen weights, expands its architecture only when necessary and performs every operation online, so inference is always available [6].

In its conclusions the PhD thesis also lists some open problems [1], including a unified comparison of these architectures, gradual drifts, and a way to shrink or reorganise networks that grow. These are open problems of the field, not stated goals of this thesis.

Connection to the CapyMOA project

In the CapyMOA project (May 2026) I integrated cLSTM and cPNN into CapyMOA, the open-source framework for learning from data streams, as classifiers (subclasses of capymoa.base.Classifier). I started from the reference implementation given in the course brief, the MagicNet repository, and evaluated both models prequentially on a real-world weather data stream. cLSTM and cPNN are the first two steps of the column-based family. In that project, though, drifts were known in advance: a field in the dataset marked the current concept and, whenever it changed, a column was added to the network. The reference works of the thesis include the next step, DYNcPNN, which removes exactly this assumption.

Where the work stands

The thesis has just started:

• the title is a working title;
• the specific research question is still to be defined;
• there are no data, code, experiments or results yet, and the technical stack for the experiments has not been chosen.

So far I have set up the LaTeX project on the official Politecnico di Milano template, with a bibliography of the reference works provided by my supervisor, and I have started studying them. The thesis repository is private, so this page does not link to it.

References

1. Federico Giannini, A First Step Towards Streaming Continual Learning in Dynamic Environments, PhD thesis, Politecnico di Milano, 2025 (Doctoral Program in Information Technology, cycle XXXVIII; supervisor Prof. Emanuele Della Valle).
2. Andrea Cossu, Federico Giannini, Giacomo Ziffer, Alessio Bernardo, Alexander Gepperth, Emanuele Della Valle, Barbara Hammer, Davide Bacciu, "A Practical Guide to Streaming Continual Learning", Neurocomputing, vol. 674, article 132951, 2026. doi:10.1016/j.neucom.2026.132951 · arXiv:2603.01677
3. Federico Giannini, Giacomo Ziffer, Emanuele Della Valle, "cPNN: Continuous Progressive Neural Networks for Evolving Streaming Time Series", in Advances in Knowledge Discovery and Data Mining (PAKDD 2023), Lecture Notes in Computer Science, Springer, pp. 328–340, 2023. doi:10.1007/978-3-031-33383-5_26
4. Federico Giannini, Giacomo Ziffer, Emanuele Della Valle, "Dynamic Continuous Progressive Neural Networks for Evolving Streaming Time Series", Data Mining and Knowledge Discovery, vol. 40, article 49, 2026. doi:10.1007/s10618-026-01213-y
5. Sandro D'Andrea, Growing Intelligent Networks for Streaming Continual Learning with Temporal Dependence, MSc thesis, Politecnico di Milano, 2025 (academic year 2023/24; supervisor Prof. Emanuele Della Valle, co-supervisor Federico Giannini). POLItesi record
6. Federico Giannini, Sandro D'Andrea, Emanuele Della Valle, "Don't Look Back in Anger: MAGIC Net for Streaming Continual Learning with Temporal Dependence", 2025 IEEE International Conference on Big Data (BigData), pp. 1396–1403, 2025. doi:10.1109/BigData66926.2025.11401614