Methodology, Response Rate, and Data Confidence

Meghodology

This data is reported in summary and released to the general public in both printed and electronic form.

Participants are asked to fill out the online survey consisting of 15 questions or a paper version with the same employment and continuing education questions. Questions asked in the survey include: current employment status, sector of employment, organization of employment, title of position, approximate annual salary, graduate program enrollment and, if enrolled, field of study and location. Participants are also asked if and where they held an internship.

Beginning in November of the survey year, subjects are identified by their graduation date. Those graduating the previous academic year (December, May and July) will be surveyed. Our office makes initial contact through email to one or more email addresses on file for the graduate. Those who do not respond receive up to three email reminders followed by a paper survey mailed in January.

As an incentive to filling out the survey we hold a drawing for a chance to win one of five Amazon.com gift certificates (one valued at $100, one valued at $50, and three valued at $25). The drawing is held in March.

Responses to the survey are confidential, handled by professional university staff, and stored in electronic format on a university server with restricted access. This data will be kept in perpetuity so as to compare subsequent surveys. Survey data is always reported in aggregate form and individual survey responses are available only to limited staff within the Career Development Center and Institutional Research and Analysis for additional aggregate reporting. Individual responses with personal identifying information are never singled out, released, or distributed within the university.

Aggregate data (not individual responses) is reported to the Minnesota Private College Research Foundation. All member career centers provide this data for state-wide reporting.

During the process of mailing out surveys the Career Development Center receives address update information from the United States Postal Service. These are handed over to the Development Office in order to update university records. Additional updated address information may be handed over to Development with the participant’s consent. Survey responses are not given to the Development Office or any other university department. The Development Office is the university office responsible for keeping alumni contact information up to date.

Response Rate and Sample Size

In statistics, sampling methods are used to form an estimate of characteristics of the entire population. With the post graduation survey the Career Development Center attempts to contact the entire class through email and postal mail. Over the last several years the response rate has been between 45 and 51 percent, which is the national average for a survey conducted 6 to 8 months after graduation.

The Career Development Center makes the best possible effort to contact the entire recent alumni population. Sampling naturally occurs when graduates fail to respond to the survey or cannot be reached. This sampling has been determined by the Career Development Center to be unbiased. Response rates are not related to major, gender, ethnicity, or other factors which may lead to a skewed response as all populations are equally represented. Those not counted were either not reachable or chose not to respond.

The Career Development Center has found no evidence of employment status affecting the response rate or skewing the results. During the recent economic downturn the Career Development Center saw a fluctuation in salary and employment (a 5% dip followed by an equal rebound over a six year period) with no apparent correlation to the changes in the 45-51% response rate. In fact, in 2009 (one of the most dismal years) the Career Development Center achieved its highest response rate of 51%. The Career Development Center determined that if employment and salary affected whether or not graduates returned the survey the response rate would have been lower in the years with low employment rates which was not the case.

Through email, mail, and online surveys everyone contacted had an equal chance to participate with the incentive to be drawn for a gift certificate.

Data Confidence

Results are monitored as they come in. Once the survey hits 200 respondents (typically a 16% response rate) the overall percentages do not fluctuate more than 5% either way. Added responses, however, increase the confidence level to support survey results applicable to the greater population of the class. It should also be noted that departmental response rates also typically follow the overall response rate showing that disciplines are equally represented.

For a population of 1,231 graduates (2012 survey population) a survey confidence level of 99% with a margin of error of +/-3.1% can be achieved with a sample size of 720 (58% response rate).

A confidence level of 95% is most used in political polls and other surveys.

To achieve a 95% confidence level the 2012 survey required 552 out of 1,231 responses (44.84%). The survey achieved this goal with 566 responses (45.98%).

With 95% confidence the survey reports that 95.58% of the 2011-2012 graduating class is situated (employed, enrolled in a graduate program, volunteer service, enlisted in the military, or not seeking employment) with a margin of error of +/-1.25%. The Career Development Center is equally confident that 65.19% are employed Full-Time (+/- 2.89%).

To learn more about sample size calculations visit www.surveysystem.com/sscalc.htm