Brtdku

I am using the h2o package to train a GBM for a churn prediction problem.

all I wanted to know is what influences the size of the fitted model saved on disk (via h2o.saveModel()), but unfortunately I wasn't able to find an answer anywhere.

more specifically, when I tune the GBM to find the optimal hyperparameters (via h2o.grid()) on 3 non-overlapping rolling windows of the same length, I obtain models whose sizes are not comparable (i.e. 11mb, 19mb and 67mb). the hyperparameters grid is the same, and also the train set sizes are comparable.

naturally the resulting optimized hyperparameters are different across the 3 intervals, but I cannot see how this can produces such a difference in the model sizes.

moreover, when I train the actual models based on those hyperparameters sets, I end up with models with different sizes as well.

any help is appreciated!
thank you

ps. I'm sorry but I cannot share any dataset to make it reproducible (due to privacy restrictions)

If you look at the model info you will find out things about the number of trees, their average depth, and so on. Comparing those between the three best models should give you some insight into what is making the models large.

From R, if m is your model, just printing it gives you most of that information. str(m) gives you all the information that is held.

I think it is worth investigating. The cause is probably that two of those data windows are relatively clear-cut, and only a few fields can define the trees, whereas the third window of data is more chaotic (in the mathematical sense), and you get some deep trees being made as it tries to split that apart into decision trees.

Looking into that third window more deeply might suggest some data engineering you could do, that would make it easier to learn. Or, it might be a difference in your data. E.g. one column is all NULL in your 2016 and 2017 data, but not in your 2018 data, because 2018 was the year you started collecting it, and it is that extra column that allows/causes the trees to become deeper.

Finally, maybe the grid hyperparameters are unimportant as regards performance, and this a difference due to noise. E.g. you have max_depth as a hyperparameter, but the influence on MSE is minor, and noise is a large factor. These random differences could allow your best model to go to depth 5 for two of your data sets (but 2nd best model was 0.01% worse but went to depth 20), but go to depth 30 for your third data set (but 2nd best model was 0.01% worse but only went to depth 5).
(If I understood your question correctly, you've eliminated this as a possibility, as you then trained all three data sets on the same hyperparameters? But I thought I'd include it, anyway.)

It’s the two things you would expect: the number of trees and the depth.

But it also depends on your data. For GBM, the trees can be cut short depending on the data.

What I would do is export MOJOs and then visualize them as described in the document below to get more details on what was really produced:

• http://docs.h2o.ai/h2o/latest-stable/h2o-genmodel/javadoc/index.html

Note the 60 MB range does not seem overly large, in general.