Model Averaging (cont.)
So far, we focused on single model
- Find best scoring model
- Use it to predict next example
Implicit assumption:
- Best scoring model dominates the weighted sum
Pros:
- We get a single structure
- Allows for efficient use in our tasks
Cons:
- We are committing to the independencies of a particular structure
- Other structures might be as probable given the data