Your metabolism can be retarded by a sudden drops in caloric intake. When the body senses that its in a negative energy balance it slows itself down to conserve energy. This can be countered to some extent by exercise but only to a point.

Both eating and exercise stimulate the metabolism to speed up.