I am having 1000 input attributes which I am trying to categorize into 100 categories. By training using logistic regression, how many model parameters needs to be learned? Will it be (1000*100 100) or 1000 100 ?

CodePudding user response：

Logistic regression is a binary classification model, meaning that it can only recognise one class from the other. In order to apply it to multi-class classification one needs to modify it, and there is no "one way" of doing so, there are some common approaches though:

The "most standard" way would be "1 vs ALL classification", which means you effectively build 100 logistic regression models, each recognising one class vs all the rest, in this case you have 100*(1000 1) parameters.

Another option is "1 vs 1" approach, where you build a logistic regression for each pair of classes, thus leading to 100*(100-1)/2 * (1000 1) parameters.

Finally, in principle you could train a model with just 1000 100 parameters, where each class only has its own bias, but projection is fixed, however this makes no sense unless your categories are orderable.