Post-Interview Feedback for Programs

Dear (Program Coordinator),

First:

I really appreciate that you asked for feedback, and I hope that the feedback I give is useful to you. I do not feel that there is any reason for the feedback to be anonymous in my case and so if you have any questions, please email me at ewarkus@hawaii.edu. I value experiences that create dialogue and lead to positive change and will prioritize this type of discussion over most other things.

Second:

I really enjoyed interviewing with your program. You have great people and I want to help you as much as I can. I’m going to give you the kind of feedback that would be useful to me if I was running a residency program and looking for applicants that are innovative, motivated, people-centric critical thinkers who want to make a difference (because I view the interview process as a mutual interview, so I believe the interview day is an important part of obtaining quality candidates). Additionally, I’m going to list feedback in list form so that it is easier to organize and read.

Third:

My perspective is influenced by the type of program that I am looking for, which is: innovative, led by strong leaders/mentors, flexible, logical decision making, empathy for patients and peers, high levels of resident autonomy from the beginning, useful consistent feedback from attendings and leadership, and demonstrates goals/actions that are consistent with the program’s mission/values and long-term plan.

Fourth:

I started writing this response on the plane in a word doc and it got really, excessively long. I genuinely want to help your program because it seems like you want to change, innovate and improve. I respect the people I met and interacted with while I was interviewing at your program and I want to give you an applicant perspective that is informed from the experience of having interviewed at nine other programs already. I wanted to give feedback on the things that were different about your program as well as the strengths that really stood out to me as I interviewed with you. I hope it helps and I’m sorry for writing a novel. I’m sure it will not be quite as engaging as a Silva novel, but I really appreciate you giving us the chance to share feedback with you.

1. Pre-interview social
   1. Overall impression: great residents. Resident 1 and Resident 2 came across as critical thinkers, humble, insightful and active about identifying their own weaknesses, grounded, empathetic. I respected them a lot. All residents were vocal about strong respect for leadership.
   1. Positives: impressed by the PGY-3s
   1. Possible improvements: ran out of food, difficult to talk to a range of people when in a crowded restaurant with long tables, not enough chairs. Personally, I prefer socials held at resident houses because it’s easier to talk, you get a better feel of the people and how the residents interact with each other.
2. Interviews
   1. Overall impression: I really liked the people. Felt very drawn to Dr. (Interviewer)  in particular.
   1. Positives: I really liked your all your faculty.
   1. Possible improvements: I wish interviewers had asked more personalized questions. One of my criteria is a rubric in which I rate whether the interviewer seemed familiar with my application, and personalized questions help with that. Bonus points if they were curious enough to visit the website I set up for the ERAS application. I have a bunch of general interview questions that I wish were asked if you are interested in hearing them. Please email me if so (ewarkus@hawaii.edu). I’ll probably put up a list on my website somewhere too.
3. Dr. (Program Director’s) lecture
   1. Overall impression: Many of my questions went unanswered and since interviews already happened, didn’t have a chance to ask my of them. It was difficult to distinguish what the important or unique things about the program are (the details like salary and work hours are not important to me and can usually be found online.) I try to ask interviewers what they feel makes this program unique because it gives them a chance to tell me their values and sell their program to me. Additionally, I have a list of the best innovations from each program I interview at, which I hope to bring to whatever program I end up in. I ran out of time to ask most of my interviewers that though because they had questions they were supposed to ask (I think). 
   1. Positives: Appreciate that you gave a talk. PD talk is very important to my perception of the program/leadership. Glad there wasn’t too much unnecessary material. Dr. (Program Director) really excelled in the ice-breaker/one unique thing about yourself. I wish the lecture had been more like that side of her personality. I thought that was great and it really brought the group together.
   1. Possible improvements: I notice prezi tends to leave a better impression on me than ppt presentations. Didn’t answer my questions. Didn’t feel like there was much time to ask additional questions. Felt uncomfortable asking questions because new group of people, whereas I had already made friends with my AM interviewee peers.
4. Resident Lecture
   1. Overall impression: If I come to the program, I am not coming for (the area), I am choosing you. I think the emphasis on selling (the area), did more harm than good because I am not a foodie/microbrewery aficionado so it made me think that the area might be a problem in my decision making process.
   1. Positives: Highlighted that (the area) does have a food and microbrewery scene which is important. Showed that residents go out and do stuff together.
   1. Possible improvements: I really hoped to know more about resident life and what it felt like to be a resident in the program. What are the attendings like? How tired are you? Do you have time to keep yourself centered? Do you have high or low levels of autonomy and responsibility? Do you want to moonlight? Do you feel like your education / preparedness to be an attending is lacking because you don’t have the opportunity to experience moonlighting? What was the best and worst offservice rotations in your experience? What has changed in the last few years with new leadership? What changes do you foresee happening in the future? Do you feel that the faculty/leadership have good people skills particularly in regards to their interactions with residents? If you could change one thing about the program what would it be? If you had a million dollars to improve the program how would you spend it? What patients do you look forward to the most? Which ones do you look forward to the least? Why? (The last three questions help me understand institution/system policies/biases/weaknesses in social aspects of healthcare). What is the atmosphere of the other residencies in the hospital towards the ER program? Do you feel respected? Do people turn to you on offservice rotations when they run into trouble or encounter difficult procedures? (don’t try to answer all those, but that’s the type of questions that I ask people at the dinner, on the tour, and at lunch or between interviews.
5. Dr. (Chair)’s lecture
   1. Overall impression: Good lecturer, inspirational, strong leader, not much detail—I suspect that there is an overarching vision/long term goal that determines how the program operates but I have no idea what it is or if that mission is something the residents or faculty are aware of.
   1. Positives: I understand why people speak so highly of the leadership and demonstrate so much loyalty.
   1. Possible improvements: I wish he had talked more about how the program makes decisions and what the long-term plan is because that influences how it run on a day to day basis and is a big part of my decision-making criteria.
6. AM vs PM interview groups
   1. Overall impression: I did not like having two groups of applicants because I value the group cohesion that happens over the course of the interview day. It’s important to see how residents and faculty interact and manage the group dynamics and the two different groups disrupted that flow. It created almost a competitive/uncomfortable feeling for me when I had gotten to know my group and talked to a lot of the other applicants but then we were suddenly lumped together with a bunch more people who were not friendly/familiar faces. Additionally, having the tour/lectures after my interviews made it much harder for me to ask questions (which might’ve been good for your time management after seeing all the questions that I have come up with so far…)
   1. Positives: Makes it so that you can interview more applicants.
   1. Possible improvements: I think it would be better to just have everyone there at the same time so you get a better feel of the group dynamics and it doesn’t feel so disorganized.
7. Survey
   1. Overall impression: I love that you solicited feedback. I was planning to email and ask if you wanted feedback, but this is more convenient. I don’t entirely believe that it’s an anonymous survey, but luckily, I also don’t particularly care about anonymity because I intended to email and ask if you wanted feedback anyway. I also noticed one of the cameras moving during the lecture and I keep wondering whether it was just an equipment thing that happens when stuff is turned on in the room or if the predilection for CIA/Daniel Silva novels mean that you were watching us 😉 (ps. I would totally do that if I was evaluating applicants because I think it’s important to know how people act/treat each other when they think no one is watching)
   1. Positives: Seriously, I respect you/(your program) for asking for feedback.
   1. Possible improvements: The questions and scales could be written better. The wording on the scales in the survey definitely influenced my choices and sometimes doesn’t represent my actual thoughts/feelings on the matter. The survey was saved by the little box at the bottom asking for any additional thoughts. Since I am comparing your recruitment strategies to all the other programs’ recruitment strategies, I think it would be better to solicit feedback in a way that asks “does this aspect (lecture, tour etc.) make you more or less likely to choose to come here for residency? How can we improve it?” For me, the social and the interviews made me very interested in your program, but the tour and lectures (with the exception of intern report) made me less excited about the program. Additionally, I would’ve added a section that asked for more feedback on the interviews. “What questions did you like? Which were not useful questions in your opinion? What are some of the best questions you’ve been asked this interview season? Which interviewer did you feel you connected with most and what was it that you felt was important/that you connected with them about? (i.e. EMS, research, ultrasound, shared hobbies/interests, etc.) Were there any questions that you felt were not an efficient use of interview time? What are you looking for in a program and do you feel that your interviewers conveyed the impression that this program has those qualities? What kind of things do you (as an applicant) pay attention to when you are at interviews?” (for instance, if interviewers have not read my application, I personally assume that the program is not very invested in finding the best fit in its incoming residents, and it gets a worse score on my grading sheet, with the exception being if the interviewer places my resume face down after I give it to them or don’t have a copy of my application (because then I assume they are intentionally blinded to my application, which I write on my sheet as well).) **I hope this isn’t too much feedback. It sounded like the program is genuinely interested in improving and changing and I felt connected enough to the people that I want to help. Since I have a reason behind everything I do, I assume that programs do as well and I wanted to share some of the assumptions/conclusions I drew based on my experience because I think feedback is an important part of improving. Also, it is endlessly frustrating to me that I cannot find a way to get good feedback after I interview at different places…so I’m modifying my approach as best as I can based on microexpressions and body language. So this is a “do unto others” type scenario in my mind. I am giving you the kind of feedback that I would hope to get as a program director. 
   1. One question that I didn’t connect with was: what three songs would you play at your best friends wedding? Just because there are so many other factors involved (i.e. I listen to country music and I wouldn’t play any of those because many people don’t like country. Also my best friend that I thought of is from fifth grade and I don’t even know what her favorite songs are anymore because we mostly talk on the phone) A version of the question that another program asked was: It’s your first day of intern year and you’re in charge of the next trauma being brought in by EMS. You’re waiting in the trauma by with your team, what song is playing in the room to while you’re waiting? I like that question better because A. it made me super excited to think about what it would be like to be an intern, B. it showed me that the program expects a high level of competency and gives autonomy in return, C. I had a great answer because there’s a song I wake up to every morning and used to listen to as I walked out of lab every night because that was my “wake up and be stoked for work song” as well as my “ok, now its time to go home and leave work at work” song.
8. Unique things about (your program) that I noticed and think are very important:
   1. PGY-3s showed an incredible amount of insight into their own weaknesses and seemed to be very good at asking for feedback—likely related to biannual reviews?
   1. You have EM program leadership from other programs come to talk at your conferences. I think inter-program cooperation is very important and is the only way to really push EM residency to excel as a whole. Your program is the first one that has mentioned anything like this and I absolutely think it’s a great idea. If I were a program director, I would use those speakers as resources to give input on things my program could do differently or ask what innovations have worked for their program.
   1. You have an orientation month with a retreat designed to form strong bonds from the start. This is quickly becoming a must-have criteria for my residency decision making.
   1. You are interested in doing quality research. I think that general lack of research interest is greatest obstacle to EM as a specialty and I find that the approach most programs have towards research is all wrong (in my opinion). There is a program (not yours) that lost my interest on the first interview because the faculty member with the strongest “research” background who was interviewing me told me that if a manuscript was not accepted, then you just find a journal that will accept it. This is major problem in EM and I think it harms the specialty. I am passionate about this particular point. Quality over quantity. Sloppy work has no place in patient care.
   1. Strong leadership – I hope to find the best role models possible in my residency program because I want to be a good leader in the future, especially if I am involved in starting an EM residency program in Hawaii. This is one of the most important aspects of programs to me. More important than research.
   1. You asked for feedback. No other program has done that and there is almost always some kind of feedback that I would like to give to a program, but I am afraid that I will negatively affect their perception of me if I offer unsolicited feedback. I do not want to seem presumptuous.
   1. You have strong, diverse, interested faculty with a lot of talent and a commitment to innovation and a good residency culture. I wish there had been more time to talk to them outside of the interview.
9. Miscellaneous:
   1. A huge improvement would have been to provide a safe, consistent place to put our belongings (or email us ahead of time to let us know that we would be moving constantly because I would’ve left everything in the car or at the security desk) I believe that it is very difficult to have a good experience and an accurate evaluation of people, interactions and places if your hands are full (to the point that I have sewn pockets on my pre-interview social outfits to stash business cards, pen/paper, phone, keys etc.) The constant moving gave the impression that the program did not have a solid base, that resources (including room space) were limited, and that program operations might be disorganized and inefficient at times. Additionally, the constant collecting of stuff and moving made it feel as though we didn’t have a place to settle into and that applicant comfort/convenience/experience were not considered. Finally, having a home base makes the process less stressful particularly for applicants that are traveling with luggage that they are forced to bring to the interview, not me, but there is an applicant like that at about 50% of the interviews I’ve been on. I spent time on this particular (relatively minor) point because it was unique to this program out of the ten interviews I’ve done so far and I suspect that it has a pretty big impact on applicants’ subconscious impression of your program.
   1. Another improvement would have been to talk with the residents about the goals for the tour, lunch interactions, etc. as well as discussing what kind of people you’re looking for in applicants. The general impression I got was that no one was really sure what they should be doing so it seemed a little disorganized and awkward. I think that those interactions are a chance to really find the best people for your program and then tell them why it’s the program for them. I think you guys have something special but you’re missing a lot of opportunities to communicate the important stuff and demonstrate that your residency program is an efficient cooperative team of awesome people that are moving and shaking things up in the field of EM.
   1. Another relatively minor suggestion: because I’ve been traveling with limited space, the only thing I usually keep from the residency gift bags are the high-quality pens (notable exceptions include a collapsible reuseable metal straw, a handy travel charger/adapter and an awesome USB thumb drive that could fit on an id badge lanyard and had the program info on it.) It seemed a little strange to give out bic pens just because almost every other program has some kind of customized pen for its applicants. The ones that I’ve held onto are the metal ones (they don’t break in baggage) especially if they have a rubber grip to make writing easier. Other things that I would hold onto and therefore be continually reminded of your program are: stethoscope holder, small high powered tactical flashlight (many programs give pen lights), playing cards, small stapler, sticky note pack with multiple tab colors, pens with multiple ink colors, permanent markers, surgical markers, binder clips, travel adapter for international travel (if you feel that’s a strong point and you want candidates that are interested in global/international med), laser pointers, cool socks, beef jerky, lock pick set, mini lint rollers for interview clothes, business card holders, travel tissue pack, travel mints, etc. I do not think you should give all of those things obviously, but giving at least one thing that is memorable and useful does make a difference for the applicants. I know it might be expensive, but the money spent on these tokens for applicants will hopefully pay off in the quality of your incoming interns even if it is relatively minor—I think that’s why most programs do give some small token or memorabilia.   
   1. An idea I had that might be an interesting way of finding people is to offer applicants the chance to anonymously (or not) mention whether there were any other applicants that they thought were really cool, good people or that they connected with during the day. I thought that (applicant 1) was a pretty wonderful applicant and I would be glad to be her friend/co-resident (I did not know her before the interview day). When I find great people that I connect with on the interview trail, I often encourage them to apply or interview at the programs that I am most interested in (if that’s something they want) and I contact programs that I have a good relationship with if there are people I think would be a good fit. In the end, I know I’ll match somewhere and I figure cooperation and helping other applicants/programs will end with a better fit for everyone despite the fact that I am increasing my potential competitors at the programs I like the most. An attending told me that the reason he did so well in his residency program was because he had a best friend and study buddy, named Tracy that he just clicked with early in intern year. Since hearing that advice, I have been actively trying to make sure that I’ll be able to find my “Tracy” at any program I end up at, and that is the rationale behind my attempts to connect the best people I find on the interview trail with the programs I like the most.

Cheers,

Erica

I respect when programs ask for feedback, so I would be a hypocrite if I did not do the same! I’d love to hear your thoughts in the anonymous SurveyMonkey link (below).

Copyright © 2022 by Erica Warkus