Conventional syllabi of engineering design courses either do not pay enough attention to conceptual design skills, or lack an objective assessment to show students’ progress. Creativity can be improved in novice designers by teaching them characteristics of good problem formulation. More specifically, when students are taught conceptual design in a structured way and given feedback using objective assessment, there is a positive within-subject change with respect to creativity. During a semester-long course of advanced engineering product design, we assigned three design projects to twenty five students. For each project we asked them to formulate the problems in a web-based data collection testbed. In addition, we collected sketches for all problems and a working prototype for the final project. We report the students’ problem formulation and ideation in terms of a set of nine problem formulation characteristics and the ideation effectiveness metrics respectively. There is a statistically significant positive trend in most measures with common trends across groups. There are some limitations in choosing design problems and the level of commitment required. Additionally, students become more competent during a course and often create similar formulations and ideas which can result in a convergence in measures of problem formulation and ideation. The adopted structured project-based approach in teaching conceptual design skills and providing objective quantified measures of problem formulation and ideation characteristics leads to improving students’ conceptual design skills.