How a hero’s welcome mirrors your own

• What is mentoring?
  • Consider a metaphor: The first time Spiderman meets Iron Man in the Marvel Cinematic Universe of films
  • Marvel Cinematic Universe of films does not tell Pete’s coming of age story
    • Before he gets to tell his own story he gets a lot of help from his mentor, who gives him the tools to be able to become a superhero
      • Provides him a framework for understanding how he “fits” in as a superhero
    • This is how everyone starts: embedded in someone else’s story before you can begin your own story
• Your mission: identify a mentor who fits your needs and learning style
  • Note: at no point have we discussed the actual doctoral programs to which you apply. Mentors make or break the experience, not the program itself.
• Every scholar in a Universe forms a galaxy of inter-connected work
  • Every mentor is like a galaxy in that larger universe, including independent pieces of work in that galaxy which can be considered solar systems
  • Pieces of work that stem from research questions that drive your work can be considered planets
  • To be able to combine these aspects into a universe you have to be able to combine them into a burning question, and the planets are designed to be the ways in which you test the burning question
• Apply to galaxies, not graduate programs
  • Think back to teenage years: think about one of the times you had an argument with your parents. Maybe you said something like “I wish I could choose my own parents”: Now you can.
  • Think about the future: who is going to vouch for you?
  • Greatly expands your options because now you no longer stay tethered to a discipline

How do mentors vary from each other?

• Mentors have burning questions
  • Start your search by identifying scholars who share topics of curiosity similar to your own
  • What does a burning question look like?
    • Example: Few minutes from Andy’s “lab talk”
      • CAIP is interested in how a child or adolescent assessments differ between reporters (i.e., the child, parents, teachers) which may lead to fundamentally different conclusions on who needs care and what type of care
      • Why do different informants provide discrepant reports of child and adolescent mental health?
        • Children and adolescents lead complex lives
        • One child with severe social anxiety may do better in groups where they don’t have to focus on one person, whereas another may be better one-on-one, whereas another may do fine in both situations but it comes down to the familiarity of the situation -> adolescents are quite unique in their clinical presentations, and CAIP aims to make sense of these unique presentations
• How do you discover burning questions?
  • Start a habit of reading work in journals in 4-week cycle (30 minutes a week)
    • Week 1: tables of contents, pick abstracts to read
    • Week 2: abstracts, pick full articles to read
    • Week 3: read articles selected in week 2
    • Week 4: scan the reference sections of the articles that you read to find articles you will want to read further
  • You will soon notice patterns:
    • In burning questions that catch your attention
    • In the authors of the authors and authors cited
  • Crucial questions:
    • Do these authors serve as faculty in doctoral programs?
    • Are they taking a student the year I apply?
  • Reading articles on topics that interest you will result in a list of potential mentors whose work overlaps with your interest
    • This is how you start a list of potential mentors
    • Not all of these mentors will make your final list!
• Mentors vary in many other ways
  • STAR Framework: heuristic for detecting variations among mentors on key facts
  • Size: size of the research team
    • Range
      • Large, expansive galaxy with lots of solar systems
      • Small, intimate galaxy with few solar systems
    • Size is somewhat correlated with mentoring style
      • Loose vs. tight supervision
      • Loose: less hands-on, more leeway
      • Tight: more meetings, closer link between interests
      • These are often related to how long/how much experience the mentor has
        • Looser style typically correlates with more experience
  • Time:
    • Quantity of time
    • Quality of time: efficiency in use of mentoring time
      • Kind of like a 2x2 table (someone can be high quantity, low quality)
    • Loose mentor style: tend to be high-quality, even if low-quantity
    • Tight: Best of these mentors tend to be high-quantity, quality often improves with experience
  • Area:
    • Different than size, area is the spread of solar system’s in the mentor’s galaxy
    • Narrow: do mentor’s students pursue burning questions that look a lot like the mentor’s question? Typically tight mentor style.
    • Wide: Do mentor’s students purse questions that are quite distinct from the mentor’s question?
  • Resources:
    • Tangible: material resources for conducting research
      • Access to clinical samples
      • Archival data
      • Equipment
      • Time (e.g., fMRI)
      • $ to fund you as an RA versus a TA
    • Intangible:
      • Network of former students
      • Collaborations with mentor’s colleagues
      • “Free advice”
• Homework: take this framework and apply it to the lab you’re currently in to give you practice on how to apply this framework

Finding mentors who fit your preferences

• Strategy #1: Knowing your learning style
  • Style 1: Learn better when left to figure it out myself (i.e., loose supervision)
  • Style 2: Learn better when given explicit directions (i.e., tight supervision)
  • Both styles produce great professionals
  • You can apply to mentors with both styles and see what fits for you
• Strategy #2: Search the public record
  • Mentors have tracks and this gives considerable power to prospective students
    • Do they publish articles with students?
    • Do students get training grants?
    • Do their students get good jobs?
• Strategy #3: Ask questions with objective answers (interview stage)
  • You will get the opportunity to ask the mentor’s students questions
  • Students rarely want to be candid with qualitative questions, easy for “bad news” to get back to the mentor
    • E.g., what is it like to work with the mentor?
    • You’ll often see a situation where you get information from the student’s nonverbals
  • Ask questions about objective elements of mentoring relationship
    • Time spent in regular meetings?
    • Papers they are working on?
    • Opportunities for grant writing?
• STAR Preferences?
  • Warning: you will have to make compromises when you make final decisions
  • It’s like real estate
    • Price
    • Age
    • Location
    • Size
    • You need to see which factors are more important to you
• File in dropbox folder that has a table to organize all of these factors
• What is the proof they can mentor me?
  • Go on a big google scholar spree
  • Among the publications of the mentor’s students do you see:
    • Students as first authors?
    • “Themes” in the articles?
    • Articles that build on one another?
  • What if a mentor does not have many students?
    • Do you see themes that cut across the mentor’s articles
    • What about in articles of students of the mentor’s own doctoral mentor?
• Excited by “Ins and Outs” of the research?
  • Go back to the mentor’s burning question articles in your first lit search
  • Focus on the method sections, are you excited about the prospect of being involved in
    • The scientific procedures?
    • Administering the measures?
    • Analyzing the data?
• Will you likely be the first or last student?
  • These are special instances of mentorship
  • For one or more mentors on your list, you may end up being their first or last student
  • Both can be great “training galaxies”
    • New mentors tend to have more time for meetings, great for students who value ight supervision
    • If you might be the mentor’s last student, do more searching
      • Birthing new solar systems?
      • Forming new collaborations with researchers?
• Student outcomes
  • Track down each mentor’s record, focusing on their previous students’ outcomes
  • Crucial question: does Student X have a job that I aspire to have?

Working with current mentors

• Me and My Mentor’s Galaxy Worksheet (in dropbox)
  • Allows you to assess your mentoring relationship on the key issues we have discussed
  • Burning question
    • Not yours, the mentors
    • No one expects you to map out your work soon after learning how to conceptualize your mentor’s galaxy
    • Helps you understand the mentor’s burning question to allow you see what other possibilities there are to explore within that galaxy
  • STAR factors
    • Standardizes a discussion you already had with your mentor
    • Sets the stage for what you would like to accomplish
    • May identify mismatches between what you want and your mentor’s style
      • May then think about “co-galaxies” (i.e., searching out a co-mentor)
    • Capitalizes on your mentor’s expertise in building solar systems in a galaxy
• Galaxy’s research programs
  • Examples of research programs:
    • A star, plants, and theoretical framework from a former grad student or post-doc
    • Completing the MMMG worksheet may lead to new research ideas
    • These new research ideas may represent the beginnings of your own research program
• Looking into the future: Your research program
  • Where are you in your training
    • Just at the beginning?
    • Near the end?
  • At any point in your training, it’s perfectly normal not to have a specified program
  • Conceptualize your mentor’s galaxy
• Special cases of mentorship
  • More than one galaxy?
    • You might find a “training gap” in your mentor’s galaxy
      • Some but not all of your needs are met
      • Formal: training grant applications may help you fuse co-mentoring relationships with a clearly developed training plan
      • Informal: may manifest if a viable co-mentor already exists on your campus or as a collaborator in your mentor’s network
  • Inhospitable galaxy?
    • May manifest in several ways
      • Personality clash
      • Mentor who’s unwilling to co-mentor when you need it
      • These things typically cannot be clear to you until you’re experiencing it
    • Strategies to consider
      • Consult an on-campus faculty mentor and ask them for advice on how to address the challenges
    • Possible outcomes to consider
      • Stick it out
      • Leave your current mentor
        • Important to find a new mentor first
        • This may involve learning a brand new set of procedures/skills, but it still may be worthwhile in the long run
        • Work with new mentor to set expectations at the beginning of the relationship
  • Change of plans?
    • What if you realize you don’t want to pursue a research career?
      • All of what we will cover today will still likely be useful
      • Will facilitate helping you tell the story of your training
      • Any job you pursue will want to know what you did during your time in training
      • Being able to clearly organize your ideas

Q&A

• Q: To what degree to previous training experiences need to map onto different experiences you will apply to (i.e., post bacc -> grad)?
  • A: It really depends on the mentor. For some instances, you can argue that your experience may help the researcher with their next steps even if your skills don’t map onto what they’re doing right now.
  • You can look at the potential mentor’s current grad students to see if a mentor may be open to consider your application if you don’t have related experience

Peer Review and your scholarly universe

• Your mentor is part of an interconnected network of scholars
  • Some collaborate with your mentor, and some do not
  • Evaluators: those scholars in your mentor’s universe who have expertise to understand mentor’s work but shares no conflict of interest with mentor (i.e., commentators)
• Evaluators
  • Play a key role in all of our scholarly universes
  • Evaluate you and your mentor’s work
  • Key goal: identify concrete peer review strategies for responding to peer review commentary

Peer review as a quality control system

• Evaluators do not perform their role in a vacuum, they exist in a system
  • Academic outlets (e.g., journals) that disseminate work
  • Experts who collaborate with outlets to judge the quality of work submitted for consideration
  • Very different from quality control systems used in other industries (e.g., manufacturing) because other industries tend to be much more objective
• Implications for your work
  • Goal: submit your work in such a way where you increase the likelihood of a fair evaluation
  • Peer review will display fair elements that facilitate your meeting of this goal
    • E.g., responding to peer review
  • Peer review will also display unfair elements that will pose barriers to meeting this goal
  • Important for you to understand the system as well as these fair and unfair elements
• Review process in a nutshell
  • Author(s) choose(s) journal outlet for submission
  • author(s) prepare manuscript for submission format
  • author(s) submit
  • Editor assigns someone to review
• Editor and reviewer selections
  • Editorial board
    • Editor-in-chief
    • Associate editor(s)
    • Editorial consultants
  • Editor-in-chief selects “action editor”
  • Action editor selects reviewers
• Review timelines
  • From time of submission, most reviews for psychology journals will take 2-3 months
  • Medical journals often take much less time (e.g., 1 month)
  • Review timelines may vary depending on how hard it is for an editor to “find” reviewers (e.g., during the pandemic)
• Review decisions
  • Action editor reads review and manuscript and decides to:
    • Accept
    • Reject
    • Revise and resubmit (R&R)
• The bad news
  • Reliability of the peer-review process is worse than the least reliable measure in any of your own studies
  • Reliability = correlation between two independent assessors' ratings of the same submission
  • Issues with unreliability apply to all forms of peer review (e.g., grants)
    • Cicchetti (1991): found inter rater reliability of reviews of manuscript submissions to range from .19-.54
    • Marsh et. al (2008) did the same thing with grant submissions to the Australian Research Council and found very low rates of reliability as well
• The good news
  • Once you get your manuscript to the R&R stage, you will reap the benefits of what may be the saving grace of the peer review process
• On behalf of peer review, Andy promises you...
  • If you submit your manuscript, get suggestions from peers, revise the manuscript accordingly and resubmit to the journal for consideration, the finished product will be better than the first

Peer review strategies when the system treats you fairly

• Do your detective work
  • The peer review process begins way before you click “submit”
  • This is because you have a surprising degree of control over who becomes your reviews
  • Pick journals to submit based, in part, on the editorial board
  • Who will give you a fair review?
    • Do you see overlap between your interests and there's?
    • But cannot be a collaborator
• Pick my reviewers!?
  • Yes you can! And should!
• How to select reviewers
  • In cover letter, identify 2-4 reviewers
  • Make sure 1-2 reviewers are on the editorial board
  • Example of paragraph to include on DropBox
• Accept your emotions!
  • Scenario: you got your decision email in your inbox
    • How do you feel?
    • Can you possibly make rational decisions now?
    • Answer: no!
    • You just waited 2-3 (or 6!) months for a decision
  • Peek and let it sit
    • Skim the decision letter
    • Did you get an R&R?
    • Great, let it sit for 2-3 days
    • Was it rejected?
      • Let it sit still
• Itemize your decision letter
  • Create your to-do list
  • Go to the reviews and start numbering your comments (even if the reviewers did not do that for you)
  • Your to-do list should be a template for your cover letter back to the reviewers/editors
    • In the dropbox: there is a template that CAIP uses to respond back in a R&R!
  • It is recommended that you quote comments and directly address their concerns
    • Makin broader claims and general changes give the illusion you are trying to hide something and makes it harder for the reviewer to complete
  • Your cover letter becomes the plan for how you will execute your revisions
  • Make a promise to yourself: address one comment per day
  • It’s OK to start with the easy one!
• Tips for cover letters/responses back
  • If you do this, most R&Rs will take you a month
  • In your cover letter, start with saying something nice and thanking any nice things the reviewers said
    • This reminds the editor and review of the positive aspects they saw in the paper
  • Constructively address comments (even mean ones) head-on
    • This shows that you respect the process and are willing to do whatever it takes
  • Embrace revision process with open arms
  • If you do not revise something as per commentary, you need a “solid gold” reason why you did not make the revision
    • Cite evidence to support your decision
    • State why you are unable to make the change
      • Unable to run an analysis etc.
• You need help, ask!
  • Beyond your co-authors, there is nothing keeping you from asking colleagues for advice on revising your manuscript
  • It’s OK to contact the editor who handled your manuscript but really only for questions that require their response
    • Asking if something is a deal breaker if you are unable to do it.
    • Asking for more time
• Empathize with your reviewers
  • Your reviewer is reading your paper “pro bono”
  • They’re like a free consultant or advisor on your manuscript
  • They could be doing many other things with their time
  • Make things easy for them:
    • Highlight your revisions in the submitted manuscript
    • When addressing their comments refer to pages, sections, and lines to make it clear
• Your eye for detail should kick into overdrive!
  • In the final stages of your revisions, “dot your i’s and cross your t’s”
    • Also, obsessively check those i’s and t’s!!
    • Everytime you revise it is good practice to re-run analysis
    • Sloppiness in the writing make reviewers wonder if there are mistakes when running the analysis
• “We would be pleased to make any further revisions”
  • Peer review = war of attrition
  • You always want to make the editor know that you are there until the end
• Be a good academic citizen
  • Just like others reviewed your work for free, return the favor and agree to review manuscripts
  • Most importantly: accept review requests in the journals were you publish
    • This will help you as you submit to these journals
  • It’s OK to email editors and let them know you are available to review (include your CV and areas of expertise)
  • You are usually ready when you have published a few times yourself which for most people is graduate school but can be as early as postbac

Peer review strategies when the system treats you unfairly

• What if two reviewers provide conflicting advice!?
  • Those are the easiest to address! Pick one
• “I would give you my kidney for more time!”
  • Good news! You get to keep your kidney
  • Just ask the editor for more times
• Sometimes you catch a mean reviewer
  • Important: no one takes a “reviewer class”
  • Reviewers are like anyone: some just have bad days
  • If it’s particularly nasty, let the editor know; it’s useful for you and the editor
    • However, they will ask if you want to restart the process and this is not recommended because there is no guarantee it will get an r&r or accept
• Do… you have a frenemy
  • Important: reviewers leave fingerprints, even if you don’t know who each other are
  • Sometimes comments give their identity away
    • They tell you to cite themselves
  • Call the bouncer
    • In your cover letter you can select a “not reviewer” and give a rationale
    • Important: take some of the blame!
      • You have different frameworks, you have a philosophical difference
      • It is also recommended that you decline to be a reviewer for their papers as well
• What if an editor rejects your revised manuscript?
  • 9 out of 10 times you should move on
    • It is very tempting to want to talk to the editor and plead your case but this will likely not change the result and hurt future chances of getting published at that journal
  • Go back and read the reviews
    • Were there any deal-breakers that you could not successfully address?
    • If yes this might mean that you should shoot for a “lower-tier” journal
    • Important: do not give up on the paper, unless you have to

Q&A

• Q: some people suggest reviewers should sign their name, what do you think of this practice?
  • A: Sometimes it’s a good thing. It can allow you to research and have a better sense of where they’re coming from, and how you can respond to make sure you’re giving justice to their comment. It’s often a personal preference thing. Sometimes there can be a power differential issue. If you’re comfortable with it it can do more good than harm. Not having to sign is meant to allow professionals early in their career and students to give their complete opinion without fear of it hurting their career.
• Q: At what point should you start reviewing?
  • A: after you publish your first paper or two. Most start in grad school. A mentor may say I have a student who’s interested in reviewing, can they review this work? If the journal accepts, the grad student acts as a coreviewer which can be a good place to start. It's important to give back to the field, but it also makes you better at your job. You can get tips on how to explain stuff better. You should not do it so much that you’re sacrificing your own work.
• Q: How do you decide which journal to submit to?
  • A: Beyond earlier advice, start looking at the reference section. You will notice a pattern in the journals that show up in the papers you cite. Those should probably be one of the ones you put on your list. For every paper, make a list of A-E journals to submit to.
• Q: Is it suggested that there is a cover letter for every submission, or is it journal-specific
  • A: every journal usually requests a cover letter. It can be helpful in a variety of ways. For instance, if you’re using a commonly used data set, you can explain why your study is unique compared to other studies using the dataset.
  • Q: Would you say that a cover letter may be a difference between an R&R and a rejection?
    • A: Maybe. Sometimes it may be the difference between the two. For some journals, those with high rejection rates, sometimes the cover letter may be the difference.
• Q: Do reviewers see the cover letters?
  • A: Yes. Reviewers always see the cover letters and the responses to comments. Sometimes if you don’t address everything, editors and reviewers might think you’re trying to hide something. It’s better to be open.
• Q: If the reviews are positive, but the editor rejects because it does not fit with the journal, should you go lower-tier?
  • A: Not necessarily. Just because it doesn’t fit doesn’t mean it isn’t good. Sometimes desk reject rates are high just because they don’t fit the aim of the journal. Every editor should reject it quickly for this reason. If they know they’ll let them down, do it as soon as possible.
• Q: Do you have any way to estimate what tier you should aim for?
  • A: Sometimes you can tell by the reference section. Sometimes it means go lower tier because what you have is a lower contribution to the field. Sometimes it’s not an incremental advance - it’s a hole that has never been filled. When it’s very novel in this way (in content, framework, etc.) go higher tier. If you have a good idea, and a study (or set of studies) and think should I split up into multiple papers or do one big paper, the common answer is to do the big paper which is more likely to get into a big journal and people will cite it more. A really good paper is like getting a golden globe, it stays with you forever.
• Q: Assuming a paper could be submitted to several journals, is it good to take impact factor into consideration?
  • Impact factor was originally created to help people (historically librarians) figure out what to buy. Now it’s sometimes used to see if someone’s work is good, should get tenure, etc. But it can give a rough estimate on if a piece of work has psychological value. For example, Psychological Bulletin has an impact factor of 16, and Psychological Review that has about half the impact factor is still like our Oscars. Impact factor can be a marker of the kind of hell you endure to get a paper published. The top tier review processes are often enormous. It often takes years to get through the review process and have it published.

Part I (Where did this workshop come from?)

-An anonymous public forum suggested this workshop

Part II (Going back to the beginning)

Mentor Reflections

-Your mentor and the relationship built with them culminates in how you conceptualize your next steps

• Filmmaking
  • -Successful films tell compelling stories
• Film Industry: The NIH of Storytelling
• Just like the NIH, the film industry invests in data to help predict whether a movie will “sell”
• Similarly, research is risk averse
• Effective storytelling
  • Best way to avoid funding a film that bankrupts a study
• What every great story does:
  • Create a world for the audience that draws them into the world/story
  • Introduce a conflict that disorients that world
  • Devise a resolution to the conflict, bring things back to normal
  • These principles are the foundation of the tools we will learn today
• Our research programs
  • Our mission: write and tell our own story which extends from our mentor’s story that shows a connection
  • Not easy: two challenges

Part III (Two potent filmmaking tools, and how to put them to use)

• The challenge, part 1
  • Your mentor’s lab is a “rogue planet”
  • This element of your mentor’s lab may sometimes feel stifling
  • Silver lining: focuses your attention
  • In an ideal world your story pivots of of your mentor’s story
  • Example: Andy’s graduate student, Bridget Makol (materials on dropbox)
• Tool 1: Shared Universe Tool
  • Your research program starts with articulating the motivations underlying your work (i.e., your burning question)
  • Practice your burning question
    • 30 second elevator pitch
    • If you do not have one yet, practice with your mentor’s
  • How does the burning question connect to the rest of the program?
    • Mentors lab = galaxy
    • Your research program = a solar system in the mentor’s galaxy
    • You need a star (i.e., a burning question)
    • You need planets (individual pieces of work that illustrate how you use the tools learned during your training to study your burning question)
  • What do (you) have to do with this?
    • The concept of the shared universe: individual movies share their own story, but each of those movies is like an episode in a television show because it’s connected to other movies in that cinematic universe
    • Think of the movies like a shared universe
      • Individual stories connect to a larger story (star)
  • Theoretical Framework
    • Framework=gravitational force that connects your star to your planets (how you conceptualize a burning question)
    • Planets represent great examples of turning ideas into action
    • Framework could be: developed by your mentor, be a well-known framework from another lab in the universe, or your own framework
• The Challenge, Part II
  • Building Your Solar System
    • What if it only has one planet?
      • Zero evidence that you can build a research program
      • Your solar system should highlight your ability to connect ideas and findings
    • How do you do it in a few years?
      • Your mentor might have had decades to construct a galaxy!
      • What tool facilitates telling a short yet coherent story?
• Tool 2: Trilogy
  • A storytelling philosophy
    • Wes Craven (horror filmmaker)
      • Background in psychology, behavior
      • Films pulled for emotional involvement, you don’t just see a horror film, you interact with it
    • To get through the aversive emotionality of horror films, Craven had the audience want to “band together” to survive
      • Audience works together to get through the end
  • Your research program is no different!
    • Your goal is to develop a story that evokes positive emotions in your audience and makes them root for you!
• First things first
  • Planets (papers) in your solar system should be around three movies in a trilogy
  • Craven’s Trilogy Guidelines:
    • 1. Make people care
      • Deep character development (audience commits to character’s well-being, bad news get the audience hooked to see what happens to these characters)
      • Filmmaker then uses that focus on the characters to their advantage (build on the audience’s commitment to characters with development to build up to a conflict)
      • Look back at your projects, which one first made you excited about your burning question?
    • 2. Introduce a conflict
    • 3. Use the element of surprise
    • Example:
      • Bridget’s Movie 1: Long history of discrepancies manifesting among informants’ reports of youth mental health (likely with mental health concerns that are difficult to discern such as depression)
        • Bridget’s study leveraged novel tools for identifying patterns of these assessments
        • Tools allow accessors to link patterns to important clinical domains relevant to assessing and treating youth depression (comorbidity, service use)
        • Setting up Movie 2: What will you capitalize off of the first movie for the second phase to create a conflict? Insert a twist to also help Movie 3
      • Bridget’s Movie 2: Discrepancies often manifest among reports of youth mental health taken in inpatient settings (often leads to doubt on reports)
        • Bridget’s study extended her Movie 1 plot to inpatient assessments taken at intake
        • Patterns in these assessments predicted treatment characteristics during stay (length, use of medication, use of locked door seclusion)
      • Bridget’s Movie 3: Reveals Movie 2 conflict: Why are we waiting for the discrepancies to come to us? What if you could force discrepancies among reports?
        • If you choose your informants carefully, discrepancies can enhance prediction, facilitate decision-making (informants are like satellites triangulating on a target, who vary in their vantage point)
      • Take home message: triad should vary in context of observation, perspective of informant
        • Powerful technique: increment prediction of observed behavior (.47-.67), incremental prediction of referral status (.27-.69)
      • Epilogue
        • Trilogies in research programs differ from movie trilogies in one important way
        • People have to know your research is not over
        • Bridget: all of the work presented focused on sample-level research using multi-informant reports, how might one use multi-informant reports to make sound decisions in…(clinical work, individual cases?)
          • Answers to these questions may require personalization
      • Element of surprise: When it comes time to prepare your research program, all of your preparation is done post-hoc, retrospectively
    • For each guideline, we will pair it with an example of scholarly work that meets the guideline
    • What’s your trilogy?
    • HW
      • Get people together to view your movie and help you work on it

Part IV (Concluding comments)

• If you understand your work well enough to be able to apply all these components of storytelling, you know your work well and can explain it to those who are not experts in your field
• Storytelling allows you to talk about and share your findings with others

Part V (Q and A)

Q: How do you tie together research from different labs?

A: You don’t have to, you only need to take a subset of the work that you do, it's okay if it crosses mentors or not, and with your work (epilogue) you can take your work and share it with other galaxies if you want. If it’s completely disconnected you can still find work that relates to you. You can mention it in the talk as another area of interest in which they can ask questions about at another time most likely at the beginning.

Q: If I have only done work on social psychology, how can I make research interesting/applicable to mentors of clinical psychology?

A: Some mentors want similar backgrounds, but many mentors accept students of diverse experiences/doesn’t completely overlap with what their labs/work are about. It is more important to be able to understand and apply science, than your background. Your diverse experiences will make people want you in their lab too. The ability to use principles and theories from different areas and apply it to your work is a valued skill. Ex. Using IAT to look at risk of suicide and suicidal tendencies was very helpful in clinical research

Q: I know entering academia is competitive, would a great story with more publications make you more valuable than one with less publications?

A: The publication amount is negligible compared to the narrative capacity of your work in the laboratory. Your storytelling ability also serves as a way to demonstrate teaching ability.

Q: When you’re looking for collaborators, do you have suggestions/tools to find students with similar interests or who are going in their own direction?

A: Mentors like to look into a student’s academic background and interests and make sure they seem like a good fit. You can do the same by looking at past graduate students and their prior interests and their work during the program. In some cases, it may be that the student is able to bring new directions to the table. Ex. part of a commonly used data set of the lab contains information about sleep, but the lab does not predominantly work on sleep. Bringing in a new student to meet that need is important to the lab.

• Learning Objectives
  • 1. Apply a Creative Commons license to at least one product
  • 2. Identify three advantages to using Wiki for disseminating info about emotional and behavioral issues and resources
  • 3. Locate at least three free resource kits on Wikiversity that can be used in research, teaching, or clinical practices
  • 4. Create a network to share suggestions and get updated versions or clinical practices
• Overview
  • Creative commons & copyright
    • Better control of our work and our sharing
    • CC, Public domain, and copyright
  • Pre-publication
    • Lab web pages
    • OSF.io
    • FigShare
    • PsyArxiv
    • Wikimedia
  • Publication options
    • Traditional (and improved traditional)
    • Wiki Journals
    • Wikiversity
    • Wikipedia
  • Dissemination options
    • Zotero, Mendeley
    • Infographics
    • Altmetrics
    • Twitter, Instagram, FB…
  • HGAPS
    • Project rubric

Copyright and Creative Commons (Licensing and Ownership)

• Economic implications and it levels the playing field
• Copyright
  • Different than patent
  • Copyright
    • Intended to protect ownership rights of creator
    • Covers writing, artistic works
    • Different laws for different countries
  • Patent
    • Covers processes and devices
• USA Copyright is complex!
  • It starts with  creation of work, even if you don’t right copyright on it
  • Lasts until 70 years after author dies
  • Different rules for “works for hires”
  • Work created by US government employees is exempt
    • Not the specific employee’s intellectual property
    • Public work
    • Key Tensions
    •  Ownership
      • Idea vs. medium (books, DVDs, etc.)
    • Registration
      • No complete directory
    • Profit
      • Author vs. publisher (or pirate), who gets it?
    • Enforcement
      • Since there are different laws around the world, international copyright laws further complicates things
      • Copyright.gov
        • You can write copyright at the bottom of your thesis for example, or you can pay $45 to register with the US government (dr. youngstrom doesn’t recommend this because there are some issues)
  • How this usually has worked in academia
    • We come up with the idea
    • Do the work
    • Write the paper
    • Send to a journal
    • Volunteer colleagues at journal review it
    • If accepted, sign a “copyright transfer agreement” (publisher now owns it)
    • Publisher makes profit
• There is lots of profit in academic publishing
  • 35% to 40% margins
    • More than big pharma, apple, tobacco
• Major Changes
  • Internationalization
    • Long history of stealing from each other and expropriation
  • Technology
    • Printing press
    • Digitization
      • What used to be illegal and difficult and expensive now is just illegal, (cheap and easy)
      • Ex. Napster wars, digital rights management (iTunes as breakthrough)
  • Open access
    • Initially perceived as fringe, now mandatory in many scenarios
• Arguments for open licensing
  • Information wants to be free
  • Reach: many authors want to reach the most people possible
    • Access to low income families, countries etc. by eliminating cost barriers
  • Lessig advocates for decriminalization of amateur activity

• Aaron Swartz
  • Postdoc at MIT, co-founder of Reddit, internet activist
  • He put resources up on to a website without a paywall (found guilty of piracy, 35 years in prison, committed suicide)
• Four elements of the creative common license → these all tell the public exactly what is allowed and how to do it! (solves the problem of contacting for permission)
  • BY: “by” requires attribution
    • Letting other people use it, but you must cite original author
  • SA: Share Alike- any remix or derivative need to use same or compatible license (only can use the same or newer version of license)
    • Your allowed to use it, but whatever you use it must be free
  • ND: No derivatives- only verbatim copy allowed to be used
    • Quote verbatim if you are going to use, not allowed to change anything
  • NC: No commercial use allowed
    • No commercial use, cannot charge for it
  • These get mixed to make licenses
• CC has 6 licenses and 2 public domain tools
  • CC-BY-NC-ND
    • Most restrictive CC license
  • CC BY
    • Least restrictive license
    • If you want academic credit, this is the one you should consider first (if you use a public domain tool (CC0) people can use your work without even giving you credit)
    • Most permissive with people still giving you credit (Dr. Youngstrom gravitates toward this license)
  • CC BY-SA
    • Wikipedia
• CC licensing does not prevent copyright
  • Copyright is basically ownership and CC is saying what is okay for other people to do with your work
  • You can have both at the same time
  • CC license does not restrict copyright owner- it is granting a license to other users ^
  • Solves practical problems of finding author and obtaining their permission
• Order Matters
  • CC-BY first:
    • The first one to consider for academic purposes
    • Fine to cite in peer reviewed articles, books
    • Free for published to use
    • Doesn;t force them to change their business article
    • (CC-BY-SA) would force them to give away book/journal!)
  • Traditional publication first:
    • Now published owns it, and we have to ask permission
• How to do it (easy version)
  • You say to yourself “ I want to give this away, but get tenure!”
  • CC BY 4.0 (write name)
  • Done!
• How to do it (hi tech)
  • USe the CC license wizard
  • https://creativecommons
  • Helps pick license
  • Gives it a logo
  • Embeds code (“watermark”)
    • Prevents removing attribution
    • Lets computer searches find object by license
      • Easier to find the free stuff and re-mix!
• Ways we could work efficiently together (goal: Exposure)
  • Archiving/ Publishing/ Collaborating
    • OSF.io
  • Disseminating
    • BitLy, QR Codes
    • Wiki- Wikipedia, Wikiversity
  • Sharing
    • Data
    • Code
    • References

Open Access (Using Open Resources “Future Proofs”

• Open Science Framework (OSF.io)
  • DOIs (OSF can give you a DOI before peer reviewing)
  • Strong version control
  • Free!
  • Future proof
  • Can chare
    • Across institutions
    • With nobody
    • With everybody
  • Problems:
    • People don't know about it or understand the ways they can use it

Wikipedia-integrated Medical Journals (A Key Health Literacy and Outreach Platform)

• All Wiki text is under the CC license
  • Share- copy and redistribute the material in any medium or format
  • Adapt- remix, transform, and build upon the material for any purpose (even commercial)
  • Attribute- credit must be given (link to the license, and indicate any changes)
  • Share alike- if you do reuse this information, it must be distributed under the same license
• Images are also Creative Commons by default
  • Optionally- remove share alike requirement
  • Remove all requirements (full public domain)
• Be careful not to violate copyright when adding to Wikipedia
  • Plagiarism detectors monitor all edits (TurnItIn)
• QR Code
  • Instead of typing in a  hyperlink, take out phone and scan the QR code
  • How to use it:
    • iPhone: point phone camera at the image; aim it so that it gets the corners
    • Android:
      • 1. Download free app
      • 2. Launch app
      • 3. Point phone camera at image….
• Bit.ly
  • You can use this to shorten your hyperlink and see the medium in which people got to the website
• HGAPS.org (Helping Give Away Psychological Sciences)
  • Nonprofit 501c3 (Feb 2018)
  • Student Service Club:
    • UNC-Chapel Hill
      • 35+ undergraduates
      • 3 graduate students, and 2+ post-baccs
    • Appalachian State
      • Group of graduate students
    • University of Maryland
    • UCLA, others ….
• The Vision of HGAPS
  • Bring the best information about psychological science to the people who would benefit and keep it free
• Making assessments in qualtrics
  • Goals of Assessment Center:
    • 1. Make it easier for clinicians to use EBA
    • 2. Improve public access to information about mental health and getting help
  • HGAPS undergraduates have coded over 65 total assessments into Qualtrics
  • Made by a group of 10 undergraduates over the last 2 semesters and summer of 2019
  • Assessments are “best of the free” as picked by experts on different disorders/symptom areas
  • Assessments go on professional society websites
• Designs of Each Assessment is Consistent
  • Introduction, instructions, disclaimer
  • Demographics section
  • Questionnaires itself
  • End screen with: (1) PDF out of the results, (2) Calculated scores, (3) resources for general public, & (4) tools for clinicians

Wikipedia As a Key Public Health Tool

• Use of the internet for medical info
  • 72% of internet users report looking online for health information within the past year
  • 53% state that it influenced their decision
  • 35% did not follow up their searches by visiting a clinician
• Competing sources of medical information
  • Of the many competing sources, most are non-commercial
  • Wikipedia’s med content dominates most search engine results
• A brief history of Wikipedia
  • 2001 began
  • 2007 editing peak
    • But poor accuracy
    • Stricter standards lead to fall-off in editors
  • 2015 plateau
    • Concerted recruitment
    • Easier editing tools
    • First year since 2007 with editor growth
  • In 295 languages
  • 5th busiest website
• Who read’s Wikipedia’s Medical Content?
  • General public
  • Medical students
  • Doctors
  • Research scientists
  • Patients, journalists, lawmakers, etc.
• Article quality
  • Articles are rated by impotence and quality
  • Quality comparable to Encyclopedia Britannica
  • Rapid sharing, rapid revision based on feedback
  • Accuracy varies by topic, but broad trends
    • Inconsistent coverage
    • Missing/ out of date information
    • Missing illustration
    • Difficult readability
  • Accuracy has immediate, real-world impact
    • Internet medical data influences the healthcare decisions of >50% of readers
    • Many articles are read a million times per year
  • Yet it has been consistently difficult to engage academics, experts and health professionals
• How is Wikipedia ruled?
  • Democracy at the top
    • Elected committees
    • Content by consensus
  • Bureaucracy
    • Policies and guidelines
    • Manual of style
  • Anarchy
    • Full volunteer
    • Extremely flexible system
    • No centralized task delegation
• Recap: Advantage to WIKI first
  • 1. Free
  • 2. Impact
    • #5 on internet
      • Works closely with #1
    • Reaches the world
    • Siri, Alexa, Google all use
  • 3. Resources
    • Can have color and links
    • Animations
  • 4. Speed
    • Can publish immediately and continuously make edits
  • 5. Transparent
    • Edit history is shown , track changes forever
    • Shows which account made edit
• EPA pages on Wikiversity
  • Goal: Make it easier for clinicians to use EBA by
    • 1. Organizing resources
      • In sequence working with client
      • Specific disorders or issues
      • Case examples
  • Giving free access to tools
    • PDFs, Translations, free scoring
• How are we paying for this?
  • Thousands of hours of volunteer time
    • Mian-Le Ong, PhD: APA “Citizen Psychologist” 2018
    • More than 300 student and volunteer editors
  • Supported by grants from SSCAP, APS, SSCP, APA CODAPAR & D12
  • Using free platforms
    • Wikimedia
    • Open Science Framework
• Recap: Online Assessments
  • 1. Assessment Center for general public
    • “Core batteries” covering common issues
    • Parent, youth, adult
  • 2. Assessment Center on HGAPS.org
    • Core batteries
    • A la carte menu of 65 more measures
• Recap: Free Resource Kits
  • 1. High School Mental Health on HGAPS.org
  • 2. Assessment Center for general public
  • 3. Assessment Center on HGAPS.org
  • 4. 200+ pages and PDFs on Wikiversity EBA
  • 5. Gallery of infographics
  • 6. Coping with … web sites
• Remix 1: Responding to Crises
  • Crisis Response Goals:
    • 1. Crowd source evidence-based information from professionals and societies
    • 2. Repackage information in digestible form
    • 3. Disseminate to those who would benefit
  • All happens as rapidly as possible!
  • Example: Responding to Crisis- Mass Violence
    • Dealing with a School Shooting page by HGAPS
      • Built after Parkland shooting
      • Crowdsourced information from psychologists specializing in community resilience and trauma inflicted on communities from disaster
      • Built infographics for dissemination on social media (primarily Facebook) to target communities and adolescents affected
      • All put online by undergraduate students in HGAPS
  • Example: Responding to Crisis Psychological First Aid
    • Psychological First Aid page by HGAPS
      • Built in concert with the Dealing with a School Shooting page after Parkland
      • Both pages give schools, parents, and adolescents tools to cope both community wide and personally with trauma
      • The curated pages and new infographics were circulated to APA to disseminate broadly to their communities
      • We made dissemination efforts on social media platforms and at UNC
  • Example: Responding to Crisis: Dissemination Efforts
    • Dissemination efforts done via social media, targeted towards the specific populations affected by crisis
      • Twitter
      • Facebook
  • Example: Responding to Crisis- Natural Disasters
    • Hurricane and Flooding Preparation page by HGAPS
      • Crowd sourcing information from psychologists specializing in community resilience, trauma inflicted on communities from disaster, and hurricanes
      • Built Wikiversity site in 2 days as Hurricane Florence hit
      • Built infographics for dissemination
      • Disseminated at local Red Cross shelter
• Remix #2: Vignettes
  • Clinical Vignettes- Purpose:
    • Dissemination and implementation science: What we wish they knew
    • Open teaching of psychological science
    • Mental health first aid
    • A beneficial outlet: Alternative to a Facebook rant
  • Clinical Vignettes- the Audience (13 Reasons Why Page)
    • Help-seeking Youth’s “peanut butter for the pill”
    • Students
    • Mental Health Professionals like Mr. Porter from the show
    • People who want to help people like Hannah Baker
  • Build a Network
    • 1. Mailing list (send updates)
    • 2. Practitioner focus groups & beta-testers
      • Feedback
      • Suggestions & additions
      • If you don’t use the tool, it doesn’t help anyone!
    • 3. Want to learn Wiki?
      • Thursday night Zooms
      • Workshop at future APA, ABCT

Outline

• Benefits to you and the student
• Recruiting
• Training
• Monitoring, Engaging, Retaining
• Mentoring
• Publishing
• Career Planning
• Lessons Learned and Resources

Benefits to You

• Builds your lab
• Establishes you as someone who is interested in working with and training students
  • Easier to recruit both undergraduate and grad students in the future
• Source of new ideas and potential new projects and publications
• Can do so much more than answer phones, make copies, enter data

Benefits to the Student

• Increase discipline-specific knowledge
• Increase persistence and success in the sciences
• increase critical thinking and org skills
• Feel more connected to campus and broader development
• Professional development
  • Meet other students
  • Letters of recommendation
• Reduces disparities in achievement
  • Empirical evidence on this
• University of Texas Empirical Study: Students who completed undergrad research had a higher chance of graduating with a STEM degree and graduating within 6 years (specifically at UT, there is an issue with people graduating after 6 years), no effect on GPA at graduation

First, some context

• De. De Los Reyes Lab
  • Recruiting and training
    • Open recruitment, draws for psych majors due to focus on adolescent social anxiety
    • All training manualized
  • Engaging
    • RAs admin all study assessments
    • Learn standardized behavioral assessments, stressor paradigms, physiological data collection
    • Some are unfamiliar peer confederates
      • Pretend they are an adolescent to see how the adolescent engages with them
    • Collect and provide data
  • Retaining
    • Skilled RA’s “promoted” to coordinator positions
    • Involved in study design and revisions
    • Supervise data collection and study assessments
    • Several then carved out a piece for independent project
  • Publishing
    • Many students from Andy’s lab published their own studies as first authors

Recruiting Undergrad RA’s

• Research Based University Program
  • Students may be assigned
  • Students have various backgrounds and motivations
• Open Recruitment
  • Advertisements and word of mouth, apply directly to lab
  • Students often have direct interest
• Examples of Research Programs
  • FIRE program at the University of Maryland
    • Research and innovation experience for first year students
  • Other campuses have enacted research programs, becoming more popular
• Open Recruitment Sources
  • Psychology classes and instructors
    • Think about “selling” your lab
    • Focus on what the student will learn and why skills are important
  • Other labs
  • Honors program
  • PSYC department list of undergrads; listservs
  • Other departments outside of PSYC like public health; other schools in the area
  • Word of mouth (why you want to be thinking about building your reputation)
  • Lab and personal faculty websites
• Once you have the person interested:
  • Have an application
    • What you might want to ask: GPA, past labs, other activities, skills they bring, time commitment (this might be the most important, do they actually have the time to commit)
    • Ask WHY they are joining a lab, and specifically YOUR lab
      • This could be a weed out question, specifically the topic of your lab
    • Grad school plans are a plus, otherwise just might be a “tourist” who are not as dedicated
    • Have screening criteria for all applications
      • Ex. GPA cutoff that can make it easier to weed out
    • Have an interview
      • It’s important to meet to see if potential RA is a good fit
      • Make sure potential RA meets with 2 members of RA
      • Discuss potential RAS with current lab staff to check for any “red flags”

Training Undergrad RAs

• Set expectations- early and often
  • You’ll be in a much better place if you do this right from the start
  • Bullet point it very specifically
• Have students read and sign a contract
  • Set policies for behavior and communication (respect, conscientiousness, accountability, timeliness etc.)
  • Outline why these policies exist (e.g. to ensure quality data collect, protect project integrity, ethical and safety obligations)
  • Describe what will happen if these policies are violated (eg. dismissal from lab, further disciplinary action)
    • Do not want people to feel like they are singled out
• Document EVERYTHING
  • Have a training manual (s)
    • Needs to be VERY specific (i.e. make sure to close the door behind you)
  • Provide step-by-step instructions for everything, and update these procedures regularly
  • Have a separate manual for “curveballs” (what do when something goes wrong or is unexpected)
  • Keep a video library of RAs engaged in research tasks (especially helpful if they are collecting data or running studies)
    • Helpful to watch other RA’s completing the tasks, ESPECIALLY if they are interacting with participant or running studies (i wrote this already wow love that)

Monitoring Undergrad RA’s

• Have an “hours per week” expectation
  • Some labs may have “open” or like “drop-in” hours but it is probably most useful to set up a specific time every week you are free to work in the lab
  • You can also do a minimum hours, (iex. you cannot work in this lab if you do not commit 4 hours)
• Set a weekly schedule for when RAs are expected to be in lab
• Track hours, task outcomes; have a central place for reporting
• Be reasonable for task completion deadlines
  • The researcher may be able to complete the task in 45 minutes, but an undergraduate may take more time than that
• Encourage RAs to spend between 1.5-3 hours in the lab each time they come in
  • Any less it’s impossible to get anything meaningful done; any more hours, fatigue and boredom will set in
• If a RA misses hours (sick etc.), make sure they schedule a time to make it up
  • This is what happens in the real world, you are expected to be accountable for that
  • You also have to tell someone you cannot show up, and how they will make up the time
• Supervise RA progress and behavior
  • Discourage cell phone and personal laptop use; consider a “backpack rule” or “cell phone cafe”
  • Lab managers/project coordinators monitor unofficially throughout RA’s time in lab
  • Use an official “check-in” form to provide RA with regular (e.g. monthly, quarterly) feedback on their performance and conduct
    • Encourage them too, especially if they are doing things right
    • Positive reinforcement
  • If an RA is not meeting expectations, set a remediation plan with very clear expectations for improvement
  • Example of hierarchical structure
    • This structure builds some incentive for undergrad RAs to become project coordinators
    • Also, it helps to delegate some of the responsibilities to the undergrads you trust
    • Helps graduate students to build important leadership skills

Engaging Undergrad RAs

• Be specific with time and task obligations- know what task the RA can do
  • Coding videos, organizing and checking data, tracking paperwork, working with participants, helping with presentations
    • Match system- what do I have to do and who is good at what
      • You may not be good at this at first
• Have regular meetings to review project goals, and expect RAs to attend lab meetings
  • Large lab meetings
  • Specific meeting to certain projects etc.
• Make sure they understand the project and their role in it- the “big picture”
  • You need to tell them more than just the task they have to complete
  • Super obvious when they interview for grad school they they did not get “science” learning and were just task-oriented
  • Make sure that the RA can answer the question: What is the research question on this study? Why are we doing this? How are you contributing to this project?
• “Meet them where they are at”
  • It is your job to find them something they CAN do
  • Play to their strengths/interests/professional and career goals
    • Note though that some students seek out these opportunities to broaden their perspectives (e.g., don’t expect that the math major only wants to work with the data)
• Provide a range of experiences and opportunities so RAs can expand their skill set
• Have benchmarks and end goals; something for the RA to work towards

Retaining Undergraduate RAs

• Consider requiring RAs to commit to the lab for at least 2 semesters/quarters
  • They could train for one semester and then leave
• Reward good performance with more responsibilities (see lab hierarchy example)
  • Staying in the lab is something to be earned
  • Consider having RAs APPLY for leadership roles within the lab
• RAs will stay if they feel valued and productive, and that they see the purpose they have in the lab
  • Again, positive reinforcement and bigger picture
• Include them on your lab website!
  • This helps build a community and a positive culture for your lab
  • Also makes them want to stay because they feel valued

Mentoring Undergraduate RAs

• For many RAs, especially young ones, this is their first professional job
  • Address anything that seems unprofessional (email greetings, communication, work ethic)
• Encourage the development of critical thinking skills- involve them in troubleshooting and problem solving
• If skill and performance allow, involve them in project deliverables
  • Seek out opportunities for them to present their work, especially if they want to go to graduate school
• Some RAs are “lifers”- they will park in your lab. This is great, but only of your research aligns with their goals
  • BUT, encourage them to get a range of experiences. They will need 3 letters of rec for most things, you will only write one
  • Support and mentor them to see out other opportunities for their goals
  • Know when it's time to say goodbye
  • Track alumni outcomes
    • Help to see where they go
    • Link people together for networking purposes

Publishing with Undergraduate RA’s

• Have clear policies on this; make sure they align with APA principals for authorship credit
• Should only be available to RAs who have been in lab for awhile and have a track record of performance, conduct, and leadership
• Best linked with undergraduate honors thesis or post bacc/ Master’s project
• Grad students may be particularly helpful here
• Be realistic about potential outcomes/outlets
  • You’re not going to send their work to a top journal
  • Be honest with yourself and the student

Helping RAs with Career Planning and Readiness

• Help them network
  • Introduce them, suggest places to apply
  • Link RAs to previous students
  • Use tracking log of alumni
• If you have time, offer to look over resume and other application materials
  • If not, refer them to Career Center
• Dealing with letters of recommendation requests
  • Create a email template with what you would need
    • What schools to to send it to and where
    • Description of why they are applying to the program
    • CV/Resume, transcript, draft of statement of interest
    • What their goals are
    • Anything else the student deems important
  • Important to think of the timing of this
    • If the deadline is quickly approaching, be honest about when you can complete it and how much advance notice you need for letters

Lessons Learned

• Lab culture
  • Positive, collaborative, and productive. Check-in on this regularly.
  • Lab retreats can help build rapport and camaraderie
• Power dynamics
  • Many at play here. Expect respect for ALL lab members
    • There will be drama between lab members
    • They interact with each other different than they interact with you
• Boundaries
  • You are NOT their friend/roommate/coworker. You are their BOSS.
    • If you started doing certain things for certain students, you have to do it for all
      • So hard to walk this back once you have started it
      • You may now be the cause of drama in your lab
      • If gender or racial difference, now you are in DEEP trouble
• Encourage RAs and lab staff to first try to resolve any problems on their own, and only then come to you if no resolution is met
• Credit hours vs. volunteers (pros and cons to both)
• Expect a range of skills, performance, and person and professional development
• Know when it's not a good fit, and don't be afraid to turn away ot dismiss, have procedures for this
  • Not everyone who wants to be an RA, can
• Don't expect problems, but be prepared if they occur
  • Some undergrad RAs may have never worked in a professional environment before, or may be living away from their support for the first time
  • RAs may be struggling and that spills over to work
  • Be supportive and understanding, but fair to other RAs and staff. Hold to your expectations and link RA to resources. Be sure to follow up.

Resources

• Undergraduate research office
• Research offices and programs on campus
• Department chair and directors
• Colleagues
• Ombuds office, legal department, Title IX office, Human Resources
• Counseling center, crisis lines, mental health resources

• Who does research in schools and why?
  • People from across fields- policy, special/general education, learning sciences, pediatrics, psychologists, allied health fields
  • Why? Schools are healthy settings that serve all children
    • It is the place to find kids
    • It’s also the place where things are launched to reach all kids
• How to think about starting up a school partnership in a new community?
  • Most new faculty members encounter a lot of questions/potential pitfalls
  • Know the culture
    • Schools have a unique culture
    • They aren’t built to do research and that isn’t their mission
    • Schools are an asset and the people there want to protect them
    • Schools can be a focus of local politics
    • Schools have their own vocabulary (PBIS- positive behavior intervention system, MTSS- multi tier support system, RTI- response to intervention, etc.)
  • Know the history-
    • How have researchers before interacted with the school?
    • Do your homework
  • Know current events-
    • Does the district face economic issues? Political ones? (e.g. strikes, curriculum controversies, etc.)
  • Know the geography-
    • May be multiple districts where you are, some may be more flexible
    • Private schools tend to have more flexibility but less diversity
  • Know district’s needs
    • If you want to have a partnership, you need to know how you can help them.
• How to think about starting a partnership-
  • Know your school-researcher network-
    • Is the district burned out on research requests?
    • Consider partnering with underrepresented communities
    • Consider how research saturation could lead to cross-contamination
      • Example- Control group may be receiving help that makes it more like another intervention
    • What can you contribute?
    • If you can branch out, it will help especially if you’re in a college town where people do a lot of research in the schools
    • Try to avoid stepping on toes of other researchers working in the school
• Beginning steps- who to contact first?
  • Who are the decision makers?
  • Know who has authority to greenlight your project
  • Learn the organizational chart- superintendent, board of education, school principals
    • May be a director of assessment, research, and accountability
    • Also could be research review or IRB committees
  • Gatekeepers are people who help you get access to decision makers
    • A kind of referral by someone who has a good reputation with the school
    • Examples-
      • Neighbor who has a long, positive history with the school
      • Your child’s teacher who is liked by the principal
      • Research colleague with existing school relationship
      • Respected school principal
    • Avoid individuals who might have a negative reputation
      • Like if your neighbor says they know a school board member, or your child’s teacher thinks you can fix the school, or a colleague who is not well connected to the school
• How to make initial contact-
  • You really don’t matter a lot to the school, and you don’t want to make it so you matter to them in a bad way
  • Want to have a presentation of “I know my work isn’t really that big of a deal to you but I’m behind the schools mission”
  • Phone call is a good way to begin
    • Use their title (doctor so and so)
    • If they are in, be very brief and agreeable- just want to arrange a meeting to talk about doing research in the schools
      • Basically scheduling your elevator pitch  
    • If they’re not in, ask how to arrange a 2 minute chat
  • Could also do a drop in visit
    • Ask to leave a one-pager, and be very brief and agreeable
  • Email- ask one question
    • Like to ask who to talk to
    • Or if you can chat on the phone
    • Or if you can bring a one-pager by
  • Some places have a research submission portal
    • Kind of like a grant submission
    • Try to ask about school/district policies and priorities first
    • Try to ask review committee chair what to avoid and what would help, and maybe even if they would look over their submission
      • Might also ask when it’s appropriate to get the IRB approval
• The initial meeting-
  • This is the first impression- one chance
  • Need to figure out how to give them something they want while you get what you need for your project
  • Some grants may ask for letter of support from districts
    • One agency is IES (Institute of Education Sciences)
  • Types-
    • Neutral-Nervous- schools are indifferent and you’re unsure how to get in the door
    • Push-Pull- schools have some interest and you’re very interested
    • Pull-Pull- both you and the school really want a partnership
  • Initial asks:
    • Learning how the district views research partnerships
    • What are next steps
    • What things should you avoid
    • Where is synergy
      • Take what the school needs and see how your valuable knowledge/experience can help
      • Dr. Cavell had a study about adolescent dating violence
        • Healthy relationships was a mandated topic for the health classes in the district
        • So they connected with a health teacher about integrating it
• Initial goals-
  • Get a foot in the door
  • Have a working alliance (getting a callback in the same day)
  • Have nobody complain about your research (teachers, parents, etc.)
  • Overdeliver
    • Provide summary of results
    • Sometimes they take muffins/tea
• Sustaining and Nurturing a Working Partnership-
  • Don’t take your partner for granted!
    • Like any relationship, think what can you give/provide
  • Point of contact (POC)
    • Ask for one POC
    • Try to communicate just with them
    • Usually a principal or school counselor
  • How to communicate with POC
    • • Assume they won’t be available when called
      • Use specific, brief language
• How to do research in schools
  • Ask little of school staff
  • Use as little space as possible
  • Make sure research activities not controversial
  • Give regular project updates
  • Get school staff to see value of project
    • Maybe give tailored findings
    • Meet teachers
    • Give incentives to teachers/staff (maybe allocate money for substitutes if asking teachers to do something for part of the day)
    • Acknowledge schools in manuscripts and send copies to school
• Recruiting participants and obtaining consent
  • Be cautious about outsourcing recruitment
    • Staff investment is key
    • Try to engage directly with students
  • In person recruitment
    • Go to the classroom and meet teachers and give a brief pitch
  • Virtual recruitment
    • Could make a short video
  • Incentivize returning of consent forms sent home with students
    • Regardless of parent decision
  • Ask about using passive consent
    • Burden is on parent to opt out
    • Some districts may be open to this
• Gathering data
  • Research team needs to be culturally responsive to needs of the school
    • E.g. having a Spanish speaker if many of the students speak Spanish
  • Measures can be online for parents
  • Make teacher surveys brief
  • Student surveys either one per period or online
  • Want to be unobtrusive if observing- don’t interrupt instructional time
• Moving to bigger projects/long studies
  • Double down on the alliance
  • Make sure there are tangible benefits from the partnership- maybe screening/mentoring etc.
• Ways to prepare graduate students for working in schools
  • Students assist in planning and conducting data collection
  • Students managing routine communication with POC
  • Accompany PI to in person meetings
  • Student writes NRSA proposals with school partnership  
• Questions/Comments-
  • Sometimes the gatekeeper is the school secretary
  • Sometimes you may modify an evidence-based intervention to better meet the needs of the school as long as it has what it needs
    • But also if you build an intervention, tailor it on the front end

GETTING YOUR FIRST GRANT: CONSIDERATIONS FOR TRAINING AND RESEARCH GRANTS

Our Purpose and Overview

• To provide info on research career development awards and project grants that will hopefully help you determine in landing grants as PI
  • Overview of career development
  • Common elements of career and project grants
    • Significance
    • Innovation
    • Pilot data
  • Review process

Territory We’ll Cover

• Training grants
• Standard NIH grants
• Unique NIH opportunities
• Foundations and private funders

Why apply?  

• Gain some opportunities that would not otherwise be available
• Receive additional mentoring and consultation
• Have protected time to work on your independent research
  • Have time saved to do own research
• Career advancement
  • Get promoted or get research done → help you in job market
• Grant options: Early Career
  • NIH:
  • T series (Institutional Training Grants)
    • Grant provides for trainees
    • Ex: When applying for postdocs in developmental psychopathology
    • Ex: T32: Ruth L. Kirschstein Institutional NRSA
      • To enable institutions to recruit individuals for predoctoral research training in a specific area (e.g. Alzheimer’s Disease, neuroscience, diversity, advanced data analytics)
  • F Series: Individual Fellowships (F Series)
    • National Research Service Award = NRSA
      • Ex: F31: Ruth L. Kirschstein Individual Predoctoral NRSA
      • Ex: F32: Ruth L. Kirschstein Individual NRSA
      • Both provide additional training you wouldn’t get in your individual institutions and to broaden scientific background in some specific area
    • Anatomy
      • Research and project information
        • Project summary/abstract
        • Project narrative (public health significance)
        • Bibliography and references cited
        • Facilities and other resources
          • For setting  
        • Equipment
      • Research and Senior/Key related people
        • Biosketches: section A should be VERY specific to the training grant and provide info regarding feasibility of sponsorship
        • Letters of support from consultants
          • Do not need biosketches
          • Meet with students and talk with students to be very specific (what are you committing to)
      • Process (tracing developmental trajectory in regards interest) → 1 page  
        • Applicant’s background and goals for fellowship training
        • Specific aims: have pilot data
        • Research strategy: significance, preliminary data, approach (method)
          • Like dissertation
        • Respective contributions
        • Selection of Sponsor and institution (where you are already and it support your research)
        • Training in the responsible conduct of research
          • Talk about ethical issues and go beyond what is business
        • Sponsor and co-Sponsor statements
        • Letters of support from collaborators and consultants
        • Description of institutional environment and commitment to training
        • Protection of human subjects
        • Data safety monitoring plan
        • Inclusion of women and minorities
        • Inclusion of children
        • PHS Inclusion Enrollment Report
        • Letters of Recommendation
  • K Series
    • Research career development awards
      • K01: mentored research scientists (for postdoc, early career person committed to research)
      • K23: mentored patient-oriented research career development award
      • K99/R00: pathway to independence award (for postdocs to transition to independent research position)
    • Anatomy
      • Candidate info
        • Background
          • Linking you to experts and critically evaluate literature that you can do your own work and interests
        • Goals/objectives
        • professional development/training plan
      • What do you need to emerge on the other side as fully independent and ready for an R-grant?
      • Specific aims: list specific objectives of the proposed research (1 page)
      • Research strategy
        • Significance
        • Innovation
          • What can change the field and fill gaps of current research
        • Approach per NIH guide
      • Candidate info + research strategy (12 pages combined)--both parts are EQUALLY IMPORTANT
      • Other sections
        • Abstract: Project Summary, also include K-specific elements described in NIH application guide (30 lines)
        • Narrative: Relevance of proposed research to public health, may also be made publicly available (3 sentences)
        • Bibliography: Provide PMICDs for citations acknowledging NIH funding and authored by applicant (no limits)
        • Facilities and Resources: Describe how environment and resources contribute to career plan feasibility (no limits)
        • Major Equipment: Describe project equipment (no limits)
        • Biosketch: attach your PD/PI, primary mentor, co-mentors and others identified as Key personnel (5 pages)
        • Current and Pending support: provide for mentor, co-mentors only. Do not include efforts or overlap (3 pages)
        • Training in the responsible conduct of research: Address NIH application guide bulleted items 1-5 (1 page)
        • Plans and Statements of Mentor/Co-mentors: Generated by each mentor, mentored proposals only (6 pages)
        • LOS from Collaborators, Contributors and Consultants: One letter provided by each non-mentor (6 pages)
        • Description of Institutional Environment (1 page)
        • Institutional Commitment to Candidate: Provide on Institutional Letterhead with Chair Signature (1 page)
  • Picking your topic and telling your story
    • Build on your graduate school focus
    • What’s missing from the picture for your, the candidate, that would transform your program of research?
      • Doesn’t necessarily need someone with a lot of publicants but someone who is really interested in research
    • What’s topical within your field (unresolved question, unmet need)?
      • How is this going to help you and the field
    • Extend to new area/skill set
    • What will you learn from this? New methods, quantitative skills, clinical trials implementation?
    • The goal is a topic relevant for you “and” the field  
    • Two main areas of focus- “lines of research” to start
    • At least one will likely be graduate school/fellowship focus
    • One may be a newer area for you
      • E.g. new passion or something innovative
    • Consider the state/depth of the existing literature
    • Ideally your two lines of work allow you to apply to different agencies/foundations
      • E.g. pediatrician and school focus or U.S. and international focus
    • This is important b/c agencies have ups and downs and shift priorities and you won’t have all your eggs in one basket
  • Consider Big Trends
    • Brain/genetic research
    • Dissemination/implementation research
    • More advanced/complex statistical techniques
      • E.g. power analyses, MLM, Longitudinal SEM, EMA
    • Disorder focus vs. developmental psychopathology
      • Risk and resilience
      • Physical and psychological health
    • Pediatric health focus vs. clinical- child
    • Do you need additional training? (e.g. workshops)
      • What skills do I need to operate more independently?

Transitioning to Post-Training NIH Funding Opportunities: R-series grants and unique (usually for your next job after grad school→ reach out to current grant funders to start writing applications)  

• Standard NIH Funding opportunities (small to large)
  • R03: 2 years, $50k/year, no budget to submit
    • Small grant program, can be useful for secondary data analysis, recording existing data (behavioral observations), and/or pilot study
    • Research strategy = 6 pages
    • No *No Cost Extensions* so plan your timeline carefully
      • Have to give money back
  • R21: 2 years, up to $275k (total; no more than $200k in any year), module budget
    • Meant for exploratory/developmental projects
      • Not a lot of pilot data and write R01 eventually
    • Research strategy =  6 pages (bare bones and description, and why to fund your project)
  • R01: generally 3-5 years, usually < $500k per year
    • Project should be clearly articulated, larger in scope than R03 or R21
      • Need preliminary data
    • Research strategy = 12 pages
      • Bigger scope and contributions
  • “Unsolicited” or “investigator-initiated” proposals (parent announcements)
    • https://grants.nih.gov/grants/guide/parent_announcements.htm
    • The types of grants above
  • Requests for applicants | Funding opportunities announcements
    • Rotate based on NIH priorities
      • Rotating calls with specific topics in mind
    • Searchable database: https://grants.nih.gov/grants/guide/search_results.htm?year=active&scope=not
• Important Info to look for in an FOA
  • Part 1: overview info
    • Have to consider who your audience is and framing your story and measures → tie into outcomes that influence
      • Participating organization: NIH
      • Components of Participating organizations
        • National institute on minority health and health disparities
        • National heart, lung, and blood institute
        • National institute on Aging
        • National Institute on Alcohol Abuse and Alcoholism
        • National Institute of Neurological Disorders and Stroke
    • Funding opportunity purpose
      • Ex: The purpose of this Funding Opportunity Announcement (FOA) is to promote research to understand the underlying mechanisms of sleep deficiencies among U.S. populations that experience health disparities and how sleep deficiencies may lead to disparities in health outcomes

Anatomy Of a Successful Application

• Preliminary Data/Track record
  • Most grants require (or at least benefit from) preliminary data
    • Shows promise for your hypotheses
    • E.g. small open trial in RCT application or strong cross-sectional associations in longitudinal study application
  • Careful consideration is warranted when picking pilot projects (e.g. start-up)
  • Pilot projects take considerable time and effort
    • Not do entire project
    • In addition to preliminary data for grant, need publications/track record
      • Feasibility for you and the study
  • A 3 timepoint longitudinal study across an important transition with measures collected on-line is not expensive and can lead to 20 publications
  • Small open trial (N = 20) vs. 30 & 30 preliminary RCT
  • Carefully and creatively select measures for your pilot work (innovation) → advancing field
• What do you mean “track record”?
  • You don’t have to be expert in everything
  • How is this going to build on what you have done?
  • Collaboration is viewed as a positive
  • Do typically need at least a few publications in the main focus area of your grant
  • Consider the newish NIH biosketch “Contributions to Science”
  • Paragraph + 4 pubs under each (want something to put there)
  • Reviewers often/always consider feasibility
  • Do you have pubs/studies under your belt that show you can pull off what you are proposing?
• Picking your team to fill in the gaps
  • Really need to know your “story” first before you can pick team
  • Consider
    • 1. Studying new constructs that are central in proposal?
    • 2. Using any new assessment/measurement techniques?
    • 3. Do you need a senior person? (Feasibility/recruitment)
    • 4. Are you going to be working in a new setting? (e.g. school)
    • 5. Do you have sufficient statistical expertise?
  • What Will My team do?
    • Varies considerably across institutions and individuals
    • Important to be clear up front what you are asking
    • Sometimes, for grantsmanship, you may bring a senior person on simply to read and edit the proposal (e.g. they won’t do any work or attend meetings if you get the grant)
    • Sometimes, you really will need them (eg. writing a statistics section, conducting a power analysis, training staff in observation procedures, making connections with schools)
    • Choosing collaborators : ask around and observe

Budget

• Hard to form a team without getting into grant budget
• Personnel (%effort) is almost the largest cost
• Percent effort should match the assigned tasks
• Substantial support/expertise- consider Co-PI
• Filling an important gap- Co-I
• 10 or free phone calls per year - consultant (rather than co-investigator to solve questions)
• Effort - 7% is about as low as you want to go & 10% feels better for someone making a substantial contribution
• Full-time coordinator vs. graduate students
• Cost vs. time vs. mentorship goals
• Consultant - $500 or $750 per day
• Co-I at 10% effort
  • 90,000 salary = 9000 per year + Fringe
  • 200,000 salary = 20,000 per year + fringe
• 9 month contracts vs. 12 month contracts (summer salary)
• PI effort may range from 15% to 40%
  • Course buyout percent
  • Trying to make it to 100%
• But who will give me money?
  • NIMH and NICD are not your only options
    • Institute of education sciences
    • SAMHSA
    • National Science foundation
    • PCORI
    • William T. Grant
    • MacArthur Network
    • Brain and Behavior Foundation (NARSAD)
    • Disorder-Specific Foundations (e.g. IOCDF, AFSP)
    • Internal Grants
    • Private Foundations

Unique NIH Opportunities: Four awards that are offered for high risk high opportunity that are not under NIH  

• Pioneer Award
  • For scientists with outstanding records of creativity pursuing pioneering approaches to major challenges
• New innovator award
  • For exceptionally creative early career scientists proposing innovative, high-impact projects
  • Part of the High-Risk, High-Reward Research Program, the award supports exceptionally creative early career investigators who propose innovative, high-impact projects in the biomedical, behavioral, or social sciences within the NIH mission
  • Must have Early Stage Investigator status (completed doctoral degree or postgraduate clinical training within 10 years and never received an NIH R01 or equivalent award)
  • Single PI only
  • No preliminary data required
  • $1.5 million in direct costs disbursed in first year of 5- year project period
  • Minimum of 25% research effort
• Transformative Research Award:
  • For individuals or teams proposing groundbreaking, unconventional research with the potential to create new scientific paradigms
• Early Independence Award
  • For exceptional junior scientists bypassing postdoctoral training to launch independent research careers
  • Part of the high risk, high reward research program, the award support outstanding junior scientists with the intellect, scientific, creativity, drive, and maturity bypass with the traditional postdoctoral training period to launch independent research careers
  • Most complete doctoral degree or clinical training between June 1, 2019 and September 30, 2021
  • Must not be in non-independent research position at time of application
  • Single PI only
  • Don’t need preliminary research
  • $250,000 direct costs per year for up to 5 years
  • Minimum of 80% research efforts in first 2 years
  • 3-5 letters of recommendation
  • Only 2 applications allowed per institution

Private agencies

• Foundations Worth checking out
  • Brain and Behavior research foundation (NARSAD Young Investigator Awards; $70k distributed over 2 years)
  • The Jacobs Foundation (early career research fellowship; %150k distributed over 3 years)
    • Usually for those in faculty positions
  • William T. Grant Foundation (yearly calls) - a lot of 25 pages
    • Call for proposals focused on reducing inequality: http://wtgrantfoundation.org/grants/research-grants-reducing-inequality
    • Call for proposals on better use of research evidence: http://wtgrantfoundation.org/grants/research-grants-improving-use-research-evidence
  • Spencer Foundation
    • Project needs to have emphasis on childhood education
  • Robert Wood Johnson Foundation
    • Current focus is on “building a culture on health”
  • Klingenstein Foundation
    • Project must have a neuroscience focus
  • Caplan Foundation for Early childhood
    • Infancy through age 7  

Review Process for NIH apps

• Reviewers rate 5 areas on a scale from 1-9
• Three reviewers/application
• Overall impact scores are averaged into a summary score
• Reviewers attend meetings and present, discuss, and provide final scores for the top 50% of applications

Review Process

• Research training plan

• • Significance
  • Thoroughness of literature search
  • Aims and hypotheses
  • Design
  • Feasibility of collecting and analyzing data
  • Pilot data
  • Power analysis
  • Analytic plan
  • Paths not taken
• Training Potential
  • Most important section; weighted most heavily on overall score
  • Feasibility of training plan (number of members on sponsorship team; very specific details about frequency, content, and duration of training opportunities)
  • Ensure training extends beyond what is currently available
  • Integration of training with research plan to facilitate development
  • Combination of coursework, lab meetings, and direct (initially supervised) research experience
• Institutional Environment and Commitment to Training (1-9)
• Additional Review Criteria
  • Protection of Human Subjects
  • Inclusion of Women, Minorities, Children (justified scientifically)
  • Vertebrate Animals
  • Biohazards
  • Resubmission
• Training in the Responsible Conduct of Research
  • Format (courses, lab meetings, individual meetings, workshops)
  • Subject matter (ethics)
  • Faculty participation (sponsor should oversee and coordinate)
  • Duration (throughout award period)
  • Frequency
  • Applications from Foreign Organizations
  • Select Agents
  • Resource Sharing Plans
  • Budget and Period of Support (feasibility; recommend as requested)

General Grant Writing and Grant Strategizing Advice

How Many Grants Should I Write?

• Ideally, have multiple grants under review
• Major caveat is that writing a grant from scratch takes a lot of time and effort
• Once you have written a grant in your core areas, work from those ideas
• Tweak sample size and scope (small and large proposals)
• Tweak focus (e.g. setting, age, disorder vs. broad)
• Overlap as much as possible with your publications
• “Program of research” is important for your story, and it also helps you be more efficient with writing

Example Proposals

• Get examples
• Colleagues and/or program officers
• Writing an R03 or R21 is very different from writing an R01 in terms of the real estate devoted to intro vs. measures vs. analyses
• Writing a grant for NSF or a foundation is very different than writing for NIH
• Examples give you estimates for how much space to devote to describing measures, analyses, etc
• Examples don’t need to be in your area, but should be similar methodologically (e.g. intervention study vs. longitudinal)
• Also wording, style, frequency, and use of citations, etc

Few Pieces of Grant-Writing Advice

1. Get Comfortable with Rejection
2. Apply Broadly
3. Get Feedback
4. Read Funding Calls and Instructions very Carefully

Overview of Applying to Academic Jobs

• June-July: Market “opens up” (but new jobs may be posted through winter!!)
  • While until posting finalized
• July-August: prep CV, research/teaching/diversity statements; obtain rec letters
• Sep-December: submit apps
• Sep-Nov: Screening Interviews
• Oct-Feb: On-Campus Interviews
  • Mostly during Thanksgiving and after new year
• Dec-April: Offers, Negotiations, and Decisions (not fixed order)
• NB: Postdoc “market” is extended winter-spring, may be pursued concurrently!

Application Package

• Research Statement (3-4 pages): may or may not tailored depending on department
• Teaching statement (1-2): teaching philosophy
• Diversity Statement (1-2 pages): how you acted on commitment on diversity
• CV (length varies, avoid “padding”)
• Cover letter (1 page, tailored for each job) - reference geographical ties
• References (3-4 pages from current and former mentors/collaborators)
• Extra resources
  • Ask for lots of examples from colleagues in your field/subfield-many are glad to “pay if forward”

What are Interviews for?

• Despite evidence of potential bias, interviews continue to be a staple of faculty searches
• Allows the search committee to obtain info unavailable through your CV, journal articles, and even letters of rec
  • Intellectual depth: how you may fit in department, and your research goals
  • Personality
  • Interest in joining their department/area

Preparing for Interviews

• Dismal Interview prep (during grad school/postdoc):
  • Attend job talks in your department (and related department); chat with your advisor (s) about each one to identify what works, and what doesn’t
• Proximal interview prep (as you apply for jobs)
  • Research each institution/department: what is their mission? Who are their students/faculty? How will your expertise, adn this new position, fit into your academic landscape? (do they need to fill gaps and needs in their department?)
• In-Depth Interview Prep (before particular interviews)
  • Think about what they’ll be thinking about and frame answers (and questions) accordingly

Screening Interviews

Basics

• From 100+ to 10-15 top candidates
• Helps decide which 3-5 are invited for on-campus interviews (these are costly!)
  • Programs want to invite those who would be successful and there is a strong chance would accept an offer
• May occur over video-conference or phone
• Typically brief (30-60 mins)
• Mostly standardized across candidates
• Usually, everyone invited for each screening have the paper qualifications

Logistical Prep

• If on video
  • Wear formal clothes
  • If possible, select a neutral background
  • Check lighting
  • Ensure strong/stable internet connection
  • Consider a headset to reduce risks for echos and background noise
• Have a sheet of paper ready for notes
• Keep a list of points to remember to bring up or questions to ask (can keep them on a post it note near the camera)
• Practice responding aloud with a friend or family member: get perspective from others  

General Tips

• Time is short, answers have to be concise, communication skills are being assessed!
• Questions to practice answering, b/c you’ll almost always get them:
  • Why do you want this position at this institution?
    • Subtext: Why would you accept this job over similar jobs?
  • Tell us about your research, where will you take it next? Future orientation should be emphasized
    • Subtext: Do you have clear plans for how you will grow your research program, secure funding, involve students, and make practical/theoretical contributions to the field?
  • Tell us about your teaching experience, what would you be able to teach here?
    • Subtext: are you able to fill our Department’s current teaching needs? → can ask what gaps they are looking for
  • Do you have any questions for us?
    • Never answer NO
      • Even though you don’t take great interest, gather all the eggs you can
    • Subtext; Did you value this interview enough to do your homework?

Tips to Guide your Answers

• Understand the “so what” of your research
  • It’s good to be excited about your contributions to the field
  • Exuberant >> Hyperactive
    • Try filming/timing yourself answering practice questions!  
• Connect your accomplishments to what you know about the job, institution and department needs
• Don’t be afraid to ask clarifying questions, better be thoughtful than right
• Express enthusiasm for the institution/job, convince them you’d really come
• Remember: they view you as a future colleague, not a trainee, present yourself and your work accordingly

Post Screening Interview

• Send a thank you email to whole search committee chair
• Last chance to express enthusiasm or clarify any answers

In Campus Interviews

Basics

• Now down to 3-5 candidates
• As noted, typically in person, though possible with video for current COVID
• Usually takes places over 2 days
• Further evaluated
• Job talk, one-on-one interviews, group meals (with student or faculty), student meetings. Sometimes also: chalk talks, teaching talk, clinical talk
• Usually, 1 person (at a time), is given an offer, sometimes 0, sometimes 2!

Logistical Prep

• Coordinate the visit logistics with administrative support team
• Make an amazing job talk, ideally start before this stage (prepare)
  • Early Career Researcher’s Toolbox has a lot of advice
• Prepare any other expected talks (institution specific)
  • Look at CVs of faculty and their research
• Read the faculty pages, CVs, and recent publications from those on your schedule. Familiarize yourself with department, college, university, institutes, initiatives, and relevant resources in community
  • Make a “cheat sheet” to reference between meetings
• Pack snacks to eat/caffeinate before your “day” officially starts

General Tips

• Be friendly! Anyone and everyone may be part of the evaluation
• These are long days, prepare with notes and reminders about the individuals you are meeting with the institutional resources
• Advice for one-on-one meetings (make it a conversation, not just question-answer-repeat)
  • 3 step process to practice with a friend or family member, may not apply to all questions
    • Acknowledge something about the question (eg. That’s something that has been on my mind quite a bit recently)
    • Answer the question (1-2 sentences, consider cues about further interest)
    • Invite their perspective (e.g. “I am curious about how you tend to think about this topic, would you be willing to share?)
  • You are someone they could work with for the next 50 years, and demonstrating care for their perspective is likely to be something someone wants in a colleague
  • Asking questions shows your interest
  • You want to avoid making an impression that you believe that you will get an offer and are evaluating them (humble confidence> arrogance---be open and curious)
  • Say aloud that you are grateful to have been invited, and highlight particular aspects of the program that you’d be honored to be part of
  • Frame your questions in a way that allows others to say “yes!” and something positive about their program, university, city, etc
    • Are they able to participate in PhD recruitment?
    • What do you like most about living here?
  • Discussions about neighborhoods and day care options (if relevant) can demonstrate you are considering moving your life

Quite note about job talk

• Read Andy’s book
• Consider whether your work builds from or intersects with work of others in department or university
• At the very least, you may be asked about the theory/work of related scholars there and it’s helpful to review this work and consider how your research connects to this body
• Your future directions can also be a place to plant seeds for potential collaborations with department colleagues
• Practice in front of friends
• Less jargon

Typical Day/Schedule --Will meet a lot of people

• Search committee chair and members
• Department faculty
• Graduate students
• Department chair
• Dean

Socializing and Meals

• Have a brief, jargon-free summary of your research ready to deliver
• Great opportunity to bond over a topic of shared interest (ie. food)!
• Consider in advance hobbies and interests that you are comfortable sharing with others, and ask about their interests outside of work as well
• Often a chance to show your interest in living in the place where your job is

Potentially difficult terrain

• Bias exists, and touchy topics include (but are not limited to)
  • What is your partner/marital status?
  • Are you looking to solve a “two-body problem”?
  • Do you have children or plan to?
  • Would you move to an area with a majority of individuals from a different racial/ethnic/religious groups?
• You may understand why they would want to know, but it’s up to you whether you share
• Plan on how you want ot address this--how you might respond to direct or indirect attempts to learn about these potential barriers to you accepting a job or being someone considered sufficiently dedicated to an academic career
• Other hard questions?
  • How much money would you need in a “start-up” package? What other non-monetary resources will you need?
    • Start considering this before your interview. Again, ask others in your network for examples
  • Would you be able to recruit your population of interest here?
    • Look up demographics of area
  • What classes would be willing/able to cover?
    • Suggest being open to intro classes
    • Learn about weather clinical supervision is expected and be ready to discuss licensure plans
  • Can you start in the fall or would you ask to defer your start date?

Other Talks you may be asked to Prepare

• Clinical Talks

Other Resources

• U Penn Career Services
• Jessica’s Full Apps
• Kate’s Full Job Application

Questions

• Q: Wear Rings and time off for kids
  • A: personal decision and look at HR website for leave and extensions and policies  
• Q: Culture of Employer
  • After you get offer, ask
• Q: Applying for faculty during postdoc?
  • Personal decision
  • Section on what to do next- expand intersections on how to do work that no one else is doing
• Q: Ask for Turnover rate? → Tenure
  • Can Google and talk to people in department
  • Thought on someone not in department
  • Why did people leave?
• Q: Spousal hires and negotiation?
  • Get spreadsheets and start up requests and ask what you needed → space and what you need to be successful b/c it’s huge topic
  • Get out of mindset that you are worth investing in even though you are below department faculty
• Q: Anything you would do differently in the app process?
  • A: more research that someone wants in the department? Are you someone that students can go to? A lot of the faculty that someone does is to provide service to the department and students
  • A: Negotiate more for your startup and advice to keep pushing

Overview

• Why a Training Grant?
• Types of grants
• Intro to F31s/F32s
• Intro to Ks
• Picking a topic and Telling your story
• What happens on the other side?
• A note about T32s
• Questions and discussions

Why apply for a Training Grant?

• Obtain grant-writing experience and exposure to federal funding agencies
• Gain some opportunities that would not otherwise be available → go beyond than your training opportunity
• Receive additional mentoring, consultation, and expand your professional network
• Have protected time to work on your independent research
• Career advancement

Types of Grants

• F30: PhD/MD programs
• F31: Predoctoral
• F32: Postdoctoral
• K01, K21: Early career award (starting faculty position)
• K99/R00: Training plus programmatic work

Other Training Grants (than NIH)

• National Science Foundation (NSF)
  • Graduate research fellowships program (GRFP)
  • Faculty early career development program (CAREER)
• Institute of education science (IES), US Dept of Education
• Foundations (Autism speaks, etc)

There are page limits

Intro to F31s/F32s

Anatomy

• Applicant’s background and goals for fellowship training
• Specific aims
  • Goal is to get other grants
• Research strategy
  • Significance of problem
  • Preliminary data: pilot tests → feasible (age range, etc)
  • Approach (method): measures, procedures, exclusion/inclusion criteria, statistical analysis
• Respective contributions
• Research and Related Senior/Key people
• Biosketches
  • Section A should be very specific to the training grant and provide info regarding feasibility of sponsorship
• Letters of support from consultants
  • Do not need biosketches
• Selection of Sponsor and institution
• Training in the Responsible Conduct of Research
• Sponsor and Co-Sponsor Statements
• Letters of Support from Collaborators and Consultants
• Description of Institutional Environment and Commitment to Training
• Research and Project INfo
  • Project Summary/Abstract
  • Project Narrative (public health significance)
  • Bibliography and references cited
  • Facilities and other resources
  • Equipment
• Protection of Human Subjects
• Data Safety Monitoring Plan
• Inclusion of WOmen and Minorities
• Inclusion of Children
• PHS inclusion enrollment report
• Letters of Rec

Intro to Ks

Anatomy (some overlap with F but some unique to K)

• Candidate Info:
  • Background
  • Goals/objectives
  • Professional development/training plan: coursework
• What do you need to emerge on the other side as fully independent and ready for an R-grant?
• Specific aims
  • List specific objectives of the proposed research: 1 page
• Research strategy
  • Significance
  • Innovation
  • Approach per NIH guide
• Research Strategy and candidate info: total 12 pages (when put together)
• Other sections
  • Abstract, project narrative, bibliography, facilities and resources, major equipment, biosketch, current and pending support, training in RCR, plans and statements of mentor/co-mentors, LOS from Collaborators, Contributors and consultants, Description of Institutional Environments, Institutional Commitment to Candidate

Curious Case of the K99

• K99 Phase: mentored (1-2 years)-------Secure Independent faculty position----> R00 phase: independent phase of career (3 years)
• Rule not to stay at same institution with K99 and R00→ some still stay at institution

Picking a topic and telling your story

• Build on your graduate school focus
• Extend to new area/skill set
• What’s missing from the picture for you, the candidate, that would transform your program of research?
• What will you learn from this? New methods, quantitative skills, clinical trials implementation?
• What’s topical in your field (unresolved question, unmet need)?
• The goal is a topic relevant for you and the field?

Review Process

• Reviewers rate 5 areas on a scale form 1-9 (1= exceptional, 9= poor), plus overall impact score
  • No formula is used to derive overall impact score from individual criterion scores; reviewers instructed to weigh the different criteria as they see fit
• Three reviewers/application
• Overall impact scores are averaged into a summary score
• Reviewers attend meetings and present, discuss, and provide final scores for the top 50% of applications (only the 50% will be brought to the large committee for consideration)

F Awards

• Fellowship Applicant (1-9)
  • Publications (esp. First-authored)
  • Letters of Rec
  • Previous Training
  • Awards and Honors
  • Grades and GREs
• Sponsors, Collaborators, and Consultants (1-9)
  • Record of funding and publications
  • Fit with training goals
  • Previous collaborations/working relationship among sponsorship team
  • Section A of biosketches and contributions to proposed work
  • Funding to cover proposed research: cover your salary and stipend not much about research project
  • Time and availability to provide training/support research from mentors
  • History of mentoring especially trainees with previous F or K Awards
• Research plan (1-9)
  • Background and significance
  • Innovation
  • Aims and hypotheses
  • Pilot data: mostly considered
  • Design
  • Feasibility of collecting and analyzing data
  • Power analysis
  • Analytic plan
  • Alternative approaches and paths not taken
• Training Potential
  • Most important section: weighted most heavily in overall score  
  • Feasibility of training plan (number of members on sponsorship team, very specific details about frequency, content, and duration of training opportunities)
  • Ensure training extends beyond what is currently available
  • Integration of training with research plan to facilitate development
  • Combination of coursework, lab meetings, and direct (initially supervised) research experience
• Institutional environment and commitment to training
• Additional review criteria
  • Additional Review Criteria
  • Protections for Human Subjects
  • Inclusion of Women, Minorities, and Children (justified
  • scientifically)
  • Vertebrate Animals
  • Biohazards
  • Resubmission
• Training in the responsible conduct of research: some institutions have requirements on training realms
  • Format (courses, lab meetings, individual meetings, workshops)
  • Subject matter (ethics)
  • Faculty participation
  • Duration
  • Frequency
• Apps from Foreign orgs
• Select Agents
• Resource sharing plans
  • Submitting data to projects NIH supported and sharing with colleagues
• Budget and period of support (feasibility; recommend as requested)

Tips

• Start Early!
  • Revise the Aims page
• Map out a timeline-- there are a lot of little pieces to do
• Get an example (funded application)
• Celebrate successes along the way  

Example Applications

• Three New F31 Sample Apps Demonstrate Successful
• Academic Twitter can be a terrific resource

What happens on the other side?

A note about T32s

• Training grants awarded to institutions, not individuals
• Predoctoral and postdoctoral
  • Predoc: Apply to program housed in your graduate institution
  • Postdoc: Apply directly to the program at prospective
  • institution
• Shameless plugs:
  • UC Davis MIND Institute Autism Research Training Program (ARTP)
  • Cincinnati Children’s T32s in Adherence/Self-Management and Child Behavior/Nutrition

Questions and discussions

• Q: Is it okay to have other sources of funding?
  • Yes
• Q: Can K or F grants be transported from one institution to another?
  • Can be; award goes to institution; variables to one taking a K award and they started off to another
• What do you recommend with a rejected application or use writing for rejection writes
  • Take a few days off of reviews
  • How do you take reviews to feasible do and what project entails and the more you can repackage things to make another product
• Can search for particular training programs or those that are funded under a training grant and search for possible co-mentors or projects; who does funding or work in that area; postdoc mentors
• Things that are helpful for you to have a more competitive application?
  • Publications are always helpful especially For Ks; for Fs, presentations at conferences are also helpful → be able to communicate it into the world; things that are under review
  • Keep ratio of poster presentations and manuscripts equal (get approval)
• How to decide a K99 and R00?
  • Flexibility to make decision later
• Timing
  • Applying to postdoc positions while doing academic year
• Tips for applying to K award
  • At institutional level, do you have time commitment to be bought out → make compelling story to make switch and your institution supports that