< JCCAP FDF < 2019

Most of the information below is up to date, however, some sections are placeholders copied from the 2018 Forum page, which can be found here. Still to be updated- grant consults description, workshop materials, notes, and all information about award winners.

Day 1 is our professional development day! We begin with a holistic set of training experiences dealing with all aspects of academic work. The experiences include a mix of interactive workshops and one-on-one and small group consultations dispersed throughout the day. We conclude Day 1 with our Forum Science Social and Forum Science Community events. These events allow presenters the chance to showcase their work in a 100% digital environment. No more than eight presentations occur at once, allowing for dynamic, lively discussion among presenters and attendees.

Day 1 also includes pocket labs and grant consults:

Pocket Labs- Members of our Professional Development Team lead small, three-person meetings where attendees receive advice on ongoing research projects. Having trouble recruiting participants? Deciding between measures to include in an upcoming project? For these and many other questions you might have about ongoing research, we are here to advise!

Grant Consults-

Block I Workshops (9:15 am-10:30 am)

Strategies for Improving Writing Clarity

Dr. Andres De Los Reyes, Ph.D.

Dr. Andres De Los Reyes is an Associate Professor of Psychology and Director of Clinical Training at the University of Maryland at College Park. His publications have appeared in such journals as the Psychological Bulletin, Psychological Review, Psychological Assessment, and the Annual Review of Clinical Psychology. He is the Editor-in-Chief of the Journal of Clinical Child & Adolescent Psychology (JCCAP), and Founding Program Chair of JCCAP's Future Directions Forum.

Description

People tend to be drawn to and understand information best when it is communicated to them in the form of a narrative or “story” rather than a list of facts. However, researchers rarely receive formal training on leveraging narrative tools when writing about their academic work. In this workshop, we describe evidence-based strategies for applying narrative structure to academic work, with a focus on preparing a manuscript for submission to a peer-reviewed academic journal.

Learning Objective

Describe strategies for using narrative tools to improve the clarity of your academic writing.

Workshop Materials
Notes
Click "Expand" for notes
  • Recommended book about writing clearly in research is Houston, We have a Narrative by Randy Olson.[1]
    • Randy Olson was a professor at New Hampshire and went off to film school and learned things that he wished he learned before in Academia.
  • Reporting fact by saying “And... And... And…” is common but there is a better way to package than this
  • Instead of the popular recommendation of writing the abstract last write the abstract or summary page first, then write the paper by expanding it.
  • If you can describe your research in 250 words or less then you can definitely unpack that.
  • Everyone writes stories with 3 main ingredients: A-B-T
    • AND (Thesis) - What you know
    • BUT (Antithesis) - Gap in the literature
    • THEREFORE (Synthesis) - What the research seeks to do.
  • Audience more likely to encode, retain, remember when in story form (instead of lists of facts)
  • You are not massaging the data or compromising science by telling a story, instead you are making sure the reader understands your message
  • Structured template to formatting most articles follows: IMRAD
    • Introduction
    • Methods
    • Results
    • Discussion
  • Most people don’t use IMRAD effectively
  • Think about one word at the core of your paper (the “cell” of the paper)
  • The word could be the theoretical framework, mechanism, mediator, etc.
    • Example: assessments of adolescent social anxiety
    • Core of paper: parents and adolescents report differently, so there is a need for a third party to assess anxiety (peers)
  • This paper’s one word: peer-reports
    • You build the paper based on this core message of peer-reports
    • Asked for a person to volunteer: create one sentence that binds all of your research interests together
  • “Nothing in…. Makes sense except in the light of…”
  • Build the story into a sentence
    • Use the ABT method to form a good elevator pitch
    • Can break your abstract into a single sentence using ABT
  • What to avoid
    • Too much complexity (despite, however, yet) (DHY)- introduces too much conflict
    • Too much information (TMI) (and and and (AAA); too many facts)
  • Words and sentences create paragraphs
    • Hardest to implement
    • Think about the story in “Hollywood style”: 3 act structure (actually utilize ABT)
    • 9 main points
      • Beginning of Introduction
      • Area of research
      • Hole in research
      • Rationale/purpose of study
      • Method
      • Results (confirm hypotheses, but study has limitations)
      • Discussion/limitations (conflict)
      • Discussions/limitations that can be addressed in future work
      • Implications/conclusions
    • Room for moving ABT pieces around
  • Activity: look at 3 abstracts, see what is wrong with each one (TMI, DHY, ABT)
    • Abstract 1- DHY- Despite, However, Yet. Contains a lot of negative statements. Too complex or too much story.
    • Abstract 2- closest to ABT of the three
    • Abstract 3- TMI (AAA) Example after example to the point it just overloaded you with information.
  • Suggestion: rewrite your own abstract in ABT form, your complete paper should be stretched out version of your ABT abstract
  • Recommendations
    • Start with your introduction in order to create a foundation with ABT
    • Your discussion should include
      • Limitations - introduce tension
      • Recommendation/future directions- resolve tension
  • Methods/Results do not need ABT because the introduction should tell a good story (readers should not forget what you said you wanted to do)
Questions from audience members
  • Are there other ways that you are trying to teach people to implement ABT?
    • Aim to create a lab at the next conference for helping people use ABT
  • The results section uses a lot of “and”, which may lose the reader- how do I combat this? Are there places to implement ABT in the results section?
    • If you see a pattern of findings in the aims that allow for telling a story, for example, conflicting findings that a secondary analysis clarifies
  • Could you use headings in the results section to help utilize ABT?
    • Don’t force a story in the results, it is okay your main goal should be to tell the truth
  • How do you paraphrase the methods section when you are using work that has been described in other papers?
    • Often you don’t want to reinvent the wheel when describing things such as methods
    • The field hasn’t found a solution for this, as you don’t want to lose clarity when describing methods that have already been well-written
    • Editors are starting to realize that plagiarism software isn’t entirely accurate, as papers are being rejected for just using the same measures in the methods
  • What is the policy for JCCAP?
    • There is no set guidelines for journals in general
    • Journals will hold their policies constant across submissions

Job Options in Academia

Dr. Susan White, Ph.D

Dr. Susan White is Professor and Doddridge Saxon Chair in Clinical Psychology at the University of Alabama. Her clinical and research interests include development and evaluation of psychosocial treatments that target transdiagnostic processes underlying psychopathology. She is associate editor for the Journal of Clinical Child and Adolescent Psychology and the Journal of Autism and Developmental Disorders, and she is the Editor in Chief of the ABCT Series on Implementation of Clinical Approaches. Her research has been funded by the National Institutes of Health and the Department of Defense. She received her PhD from Florida State University.

Dr. Matthew Lerner, Ph.D.

Dr. Matthew Lerner is an Associate Professor of Psychology, Psychiatry, & Pediatrics in the Psychology Department at Stony Brook University, where he directs the Stony Brook Social Competence and Treatment Lab. His research focuses on understanding mechanisms of and developing interventions for social and emotional functioning (in particular peer relations) among children and adolescents with autism spectrum disorders and ADHD. Dr. Lerner has received over $8 million in funding for his work from the National Institute of Mental Health, Brain & Behavior Research Foundation, American Psychological Foundation, Simons Foundation, Alan Alda Fund for Communication, Arts Connection, and Pershing Charitable Trust.

Description

Graduate training in fields relevant to child and adolescent mental health (e.g., Education, Psychiatry, Psychology, and Social Work) prepares trainees for careers in a variety of policy, research, and practice settings. How does a trainee learn about these opportunities and maximize their chances for landing jobs in one or more of these settings? We will provide attendees with a broad overview of the job options available in academia, with a specific focus on strategies for crafting the training and scholarly records that make someone a compelling candidate for these job options.

Workshop Materials
Notes
Click "Expand" for notes
  • Why stay in Academia?
    • Staying in academia may offer more work-life balance than having a career solely based in clinical work
    • A sense of impact - work such as papers and speaking at conferences can impact more people than one would ever meet
    • There are more options in academia than in practice
      • There are different settings (universities, hospitals, industry) and different career paths (consulting, research, pharmacology)
  • What are the options [for work in academia]?
    • “Traditional academic jobs”
    • research 1 institution
      • What determines if the university is an R1 institution: amount of grants received, doctorate degrees given, etc.
        • Though, not ever researcher may want to be at an R1 institution
      • These are more research intensive jobs where your job is to run a lab, train graduate students, and teach some - even if it’s not technically an R1 institution people use this term colloquially to refer to this type of job
    • 50/50
      • Half research/half teaching, half research/half clinical work
        • You rely on using your other hours than what’s used on research for other activities
    • Teaching-focused
      • More commonly found at liberal arts colleges
      • There’s still an expectation that you remain involved in research but it isn’t the main hat that you wear/the main task expected if you
      • In the process of thinking about a given job, you need to be mindful of “is this really all I’m being asked to do”- sometimes for teaching-focused jobs there are other tasks that aren’t technically under the job position
    • Research positions
      • Medical School
      • “Center”
        • Centers are built in universities to bridge disciplines
        • The epitome of “team science” and working with people outside of your discipline
        • This is a growing focus within academia
      • Government
        • Government funded research centers
  • Key issues
    • Intellectual/curiosity - Academic careers give you the most freedom to study the thing you want to study without the fear of being fired (if you are tenured)
    • Source of money - Academia likely won’t give you the most money starting off the bat
    • Flexibility - in clinical work you may be tied to your phone and be on call more because your patients are at risk
      • On the other hand in academia you may be spending more time due to hard deadlines that need to be met
    • Autonomy - You bring your expertise into the room and build your own autonomy
    • Geography - Academia will typically tie you to an institution that has more than one location
    • Helpful tip: Use a decision-making chart in order to weigh your values with each academic job option
  • To tenure or to not tenure
    • Gives you the freedom to pursue any academic research without fear of losing your job
      • Often granted after 6 years, give or take
      • The tenure track is like a probationary period and is likely the hardest critical period of your career
    • Typically tenure comes along with a promotion so it’s hard to decipher whether or not it’s associated with higher pay, or if the promotion is associated with higher pay
      • For the most part, medical schools will pay more right out of grad school but it comes with the catch that for what portion of that pay are you responsible for bringing in
  • A few words on teaching
    • Things to consider when speaking with potential employers:
      • Load options - your course load; in an R1 institution it’s most common to teach two classes for two semesters
      • Reductions - how do you reduce your teaching load for the first couple of years to help you establish your research program?
        • This is common
        • You can also negotiate a buy-out; you can buy out a part of your salary using grant money to negotiate teaching fewer courses
      • Types of courses - it’s optimal to merge what you teach and what you study
      • Efficiency - how many TAs will you have, aka how much of the workload of teaching is on you?
      • Preparation - is this a new course that your teaching/how much will you have to prepare for it?
        • If it’s a recycled class there’s less preparation time
      • What would you like to teach? Do you enjoy teaching?
  • A few works on service
    • Committee work
      • Internal - inside the university
        • Ex. working for a search committee for new faculty
      • External - outside the university
        • Ex. working for the APA on a committee
    • Leadership
      • Demonstrate that you showed leadership in some domain
    • Reviews for papers and grants
      • Doing Ad Hoc grant reviewing can lead to becoming a sitting member on a journal
        • Much more time consuming than reviewing papers
          • While a paper can take two hours, a grant takes about 3-4x that amount of time
        • Grants also feel much more consequential due to the amount of money on the line
        • This is heavily weighted on a service point of view
      • You may review for an education board
        • An editorial board is a small group of people who make determinations on whether or not manuscripts are passed
          • It is very important to be mindful of which journals you agree to be on the ed board at - Is this somewhere that fits in with your interests/you would submit a research article to?
      • What are the criteria for sitting on an ed board?
        • Ultimately, it’s completely up to the editor in chief
          • Typically they know you and reach out to you who have either published a lot in the journal or you’ve reviewed for them a lot already and they noticed you do a good job
            • This is why it’s recommended that you reach out early on in your career and ask to review for them (Guideline: If you’ve done 10 reviews, and they were good, that’s enough to ask to “serve the journal at a higher level” aka ask to be a sitting member)
      • Note: every review you do gets a grade on timeliness and quality
      • Grad-students
        • Co-reviewing: you can review alongside your mentor to help get your foot in the door while gaining the knowledge on how to do a review
        • Translational Issues in Psychological Science: an APA journal that allows graduate students to be a reviewer
      • This shows that you are trusted enough to be given the responsibility of deciding if an article is good enough to be published
    • Home organization involvement
      • Being involved in your campus, community, department, etc
    • Balancing service - knowing when to say “no”
      • It can be hard for junior faculty to know when to say yes versus no - it’s helpful to have a mentor senior faculty who can help you make these decisions
        • Finding a mentor in your institution is vital
          • Some institutions even require a certain amount of meetings with a senior faculty mentor
          • If there is no formal program, you want to decipher when you’re interviewing is if the climate of the institution is “every man for themselves” or not (if they are, it’s a bad sign)
            • Right from the get-go think of who you want to meet with while interviewing to be able to form this mentorship
  • Preparing:
    • What does the literature say?
      • What predicts employment?
        • School reputation (Carnegie) and department reputation (NCR ranking): each provides predictive power
          • Look at faculty research productivity, faculty citations, number of publications distinguished between Ph.D. holders
      • What is most predictive of employment: department reputation
    • Interests versus programmating
      • Market considerations
        • There is an increasingly competitive job market
          • 18% of graduates go into an academic career within 5 years of degree
          • ~40% go into non-academic jobs
        • There are more students, yet fewer tenure-line positions
          • Increases in non-tenure track/teaching faculty
          • From 2003 to 2013, there has been a 40% increase in the number of doctorate students
    • What are the standards for entering a career in academia?
      • Mean number of publications for a 4th-year student: 3.47
      • Professors who had formal postdoc training had ~3.67 more publications than those without
      • Newly hired faculty (with postdoc training) had a mean of 13.37 publications
        • It’s hard to quantify the quality of the publications, but it matters just as much
  • Applying:
    • Timeline
      • Spring
        • Begin thinking “Am I ready?” in terms of your CV, dissertation, etc.
          • This may adapt depending on what sort of position you’re applying to
          • A good rule of thumb if you’re going into the academic market is that you want to go “all in”
            • It’s a very time-consuming process, so if you make the decision to apply you should really do it - it’s essentially a part-time job
          • Reach out to your advisors and let them know you’ll be applying so that they’re prepared to help you and write letters of recommendation
        • Begin to put together your research statements and teaching materials
      • Summer
        • This is when job postings typically begin going up
        • Begin soliciting your letters of rec as early as possible
        • Update your CV, order your transcripts, get reprints (PDFs) of your publications together
        • Complete your research statement and seek out as much input on it as possible
          • This is even the case for some teaching-focused schools
          • This is your “calling card” for your academic scholarship that articulates your interests
            • This is what I’ve done, how I did it, and what I hope to do at your institution
            • Note: you want to make the fundability of what you want to do be clear
              • This can include saying what grants you will apply to
        • Draft your teaching statement & compile your teaching evaluations
          • Particularly if you’re in an academic department (Psychology, Education)
          • If it’s not a teaching-focused school, it will be weighted less
          • Your philosophy of teaching (it’s okay if it’s full of a lot of platitudes - most are)
            • And give examples of what you’ve gone
              • Note: mentoring undergraduate students counts
              • You can even give quotes from nice emails you’ve received from students
      • Fall
        • Mailing out applications
          • You can upload all of your files into “Interfolio” (including letters of recommendations) and then it can be directly sent to institutions from there
        • Start thinking about/draft your “job talk” which is about a 45 minute summary of your research
          • Some may want a teaching sample or “chalk talk”
        • Begin doing phone interviews and some in-person interviews
          • Phone interviews: act as an initial “smell test,” it helps to be broad and enthusiastic
          • In-person interviews: the main goal is to assess your fit
      • How else to prepare
        • Use your advisor as a resource
        • Think carefully about your priorities (job type, department type, significant others’ needs, resources, location, etc)
        • Job search resources
        • Be organized through things such as excel sheets to organize your timeline and materials
        • Talk to a lot of friends and colleagues
          • No one person you talk to for advisement will know everything - talk to a group of minds and get a holistic view
        • Practice!

Preparing a Training Grant

Dr. Deborah A.G. Drabick, Ph.D.

Dr. Deborah Drabick is an Associate Professor of Psychology at Temple University. Her expertise is broadly in developmental psychopathology, and more specifically in youth externalizing problems. Her work includes such areas as risk and resilience, co-occurring psychological conditions, contextual influences, and intervention. Dr. Drabick has been funded by the National Institute of Mental Health, American Psychological Foundation, PA Department of Health, and Temple University. She currently serves as an Associate Editor for the Journal of Clinical Child & Adolescent Psychology.

Dr. Tara Peris, Ph.D.

Dr. Tara Peris is an Associate Professor of Psychiatry and Biobehavioral Sciences at the UCLA Semel Institute, where she serves as Program Director of the UCLA ABC Partial Hospitalization Program. Her research focuses on developing strategies for optimizing treatment outcome for difficult-to-treat cases of anxiety, OCD, and related conditions. She is the recipient of a career development award from the National Institute of Mental Health, a NARSAD Young Investigator Award, and awards from the Obsessive Compulsive Foundation, the Trichotillomania Learning Center, and the Friends of the Semel Institute.

Description

Submitting a training grant involves considering multiple factors that focus on not only a proposed study but also a concrete plan for developing the skills needed to execute this study. By construction, these applications carry many expectations, requirements, and complicated forms. In this workshop, we leverage our years of experience with extramural funding to clarify the process of submitting a training grant, and provide attendees with concrete tools for submitting successful training grant applications.

Workshop Materials
Notes
Click "Expand" for notes
  • Overview
    • Why a training grant?
    • Types of grants
    • Intro to F31/F32
    • Intro to the K
    • Picking a topic / selling your story
    • What happens on the other side?
  • Why apply for a training grant?
    • Gives you opportunities not otherwise available
      • It is helpful to think about what opportunities are already available to you
      • The training grant should be focused on things you wouldn’t otherwise be able to get
    • Helps you buy out of other things you’d have to do
      • K - reduced course load
      • F - wouldn’t have to TA
    • How will this advance your career trajectory?
      • What is the next step in your research program?
    • “I already bring X to the table, but if you could only give me Y, it would be so great… etc”
  • Types of grants
    • F-30 PHD/MD
    • F31 Predoctoral
    • F32 postdoctoral
    • K01, K21: early career
    • K99/R00 training plus programmatic work
  • Anatomy of a F30/F31/F32
    • Research and project information
      • A summary/abstract
        • Describes what you plan to do and the training goals you have
        • We like it when the training goals are specific and reflected in the training plan
      • Project narrative
        • Why the work is important
      • Bibliography
      • Facilities and other resources / equipment
        • Are the tools available at the institution to do the work?
        • Want to make sure you’re not setting yourself up to fail
        • This part might seem boilerplate, but it is important to think carefully about the specifics of your study and how it interfaces with the resources in your department
    • Biosketches
      • Section A should be very specific to the grant and provide information about feasibility
        • This section should NOT be boilerplate
        • “I have expertise in A, B C, and I am willing and able to bring this to this grant”
        • If there are people at different sites, it’s important to show that the relationships are established
        • Again, feasibility is highlighted here
        • The sponsor should have experience mentoring students with the grant in question
          • If the advisor doesn’t have this experience, it can be helpful to bring in a co-sponsor
          • Their work should also dovetail nicely with the goals of the grant
          • I.e., their research shouldn’t be unrelated to the grant
        • Every person on the grant should have a unique role
          • There can be some overlap, but each person should have a needed skill (avoid overkill)
          • This shows you’re thinking carefully about the training plan
        • It can be helpful to have a table listing the mentors, what they bring to the table, and how you plan to have ongoing access and support from this person
    • Applicant background
      • Tell the story of how you got to the place where you are asking this question
        • How will this fellowship propel you on this trajectory?
      • It’s okay for your interests to evolve!
        • You just have to tell the story of how and why this happened
        • It’s also okay to state that you have multiple lines of research
    • Specific Aims
      • Focused on the research you’ll be doing
    • Research strategy
      • Why are these gaps important to address?
      • Helpful to see some preliminary data
        • Shows that this is in fact feasible (both in terms of theory and facilities)
      • Power is an important consideration
        • Prior work can help justify this section
    • NB: having information in different sections can be useful because some reviewers do not read all sections!
    • Respective contributions
      • When you wrote the grant, who helped you?
      • This helps demonstrate that the sponsorship team actually cares about the project
    • Selection of sponsor institution
      • More boilerplate information about the site
      • Training in the responsible conduct of research
    • Sponsors and co-sponsors
      • Who they’ll communicate with one another and provide support
      • This should be very specific, while also being feasible
        • It can be helpful for a mentor to say, e.g., “we already meeting once a week, and if funded our focus would shift towards…”
        • However, it’s also important to show that this grant is giving you something more than what’s already available
  • Anatomy of a K
    • What you need to be competitive for an R grant is a track record of prior funding
    • For a K-award, the typical trajectory involves some smaller foundation grants
      • These can be smallish (~20k)
      • The benefit of a K award is that it covers 4 or 5 years of salary
        • Universities cover about 15%, or there are some teaching activities
    • Even split between research proposal and training proposal
      • You should give these portions equal time and emphasis
        • It is bad if the training plan seems like an afterthought
      • They are weighted equally
        • On the F grants, the training piece get weighted more
    • Similar format to the F
      • Research strategy: significance, etc.
    • The topic does need to find a relevant place in the field
      • What are the hot topics in the field?
      • How can you show that you’re aligned with the current trends in research?
        • This is important because the next step after a K is an R, and so it’s important to show that you have a longer-term plan
    • Talking about the skills learned from the K
      • It’s good to talk about how you’re going to use your project as a platform for testing, refining a skillset that will give you a lot of longevity
    • There are many other supplemental sections to the K
      • It’s important to give yourself some time to write all the other sections
        • Abstract, biosketches, training template, etc.
    • Writing a K or an F is a good opportunity to cold-call researchers you might want to work with
      • People tend to be pretty generous with their time; if they can do it, they often will!
      • It’s good to write an abstract up-front so that you can share it with potential collaborators and program officers
      • This will help you move outside your immediate circle and market your idea
  • Picking your topic and telling your story (redux)
    • This is an exercise in being able to talk about who you are and where you’re going
      • Undergrad, post-back, graduate school -> how do these pieces line up?
      • It’s okay to show off -> avoid a self-effacing frame
        • They know that no one just handed you a first-author publication, for example
    • Much of this is about making the case for why you need this specific piece of training
      • Training can take a lot of formats
        • Didactics (classes) can be a great way to do this
      • It’s important to talk about how you’re going to bring the knowledge home and apply it to a particular project
        • E.g., we know you’re not going to be an expert in stats after a 1 week workshop, but it’s good to be able to talk about what you will take away
  • Review process
    • Reviewers rate 5 areas on a scale from 1 to 9
      • Three reviewers per application
      • Overall impact score are typically averaged into a summary score
      • Reviewers attend meetings and present, discuss, and provide final scores for the top 50% of the applications
        • Based on aggregated scores submitted before the review meeting
        • Everyone gets feedback, even if the grant is not discussed
    • Fellowship applicants
      • Publications
        • First-authorship is key
        • Travel awards, posters, etc. are great too
      • Letters of rec are important for showing initiative and background in the subject-area
      • Awards and honors
        • Includes undergrad, graduate school
        • Shows your trajectory
      • Grades and GRE scores
        • If there’s something potentially worrying on your transcript, it’s a good idea to address it in the biosketch, or even in the sponsor’s letter
    • Sponsors, collaborators and consultants
      • Record of funding and publications
      • Fit with the training goals
      • Should show that the sponsor has worked with the applicant
      • This information comes out of Section A
        • It’s important that this information isn’t merely boilerplate
    • How the funding will cover / support research
      • Should show a history of mentoring
        • Can be helpful for the advisor to list outcomes of other students, even if they didn’t get F awards
    • Power analysis
      • This part should show you how carefully you thought about your question
      • There may not be effect sizes available, so it’s important to show how thoughtful you are in considering the possibilities
      • If it’s a training plan and you have a stats person, it’s important to get their feedback on the application
    • Training potential (the most important section)
      • Feasibility of the training plan
        • How many people on the team
        • Frequency, content, and duration of training opportunities
        • That the training extends beyond what you already have
        • How the training informs the research plan
        • It’s really good to show that there’s someone helping you take what you learned and apply it to your research
    • Institutional environment
      • Letter from chair of department should show the institution’s support for the candidate specifically
      • They should know who you are and care
    • Training in the Responsible Conduct of Research
      • Should show more than just CITI training
      • What other resources are available for ethics?
        • What other formats? What content will be covered?
      • Important that the faculty member shows oversight
      • If there are brown-bag seminars devoted to ethics, it’s important to include that
    • NB: this is a place where you can get points on your application with relatively little effort
        • Again, shows attention to detail
  • Timing
    • For F series grants, which support 2 years of dissertation work, 2nd or 3rd year is a good time
    • For the K award, it takes some time to line up the pieces, but it’s good to think about putting something in within 2 years of completing the PhD


Block II Workshops (10:45 am-12:00 pm)

Responding to Peer Review

Dr. Andres De Los Reyes, Ph.D.

Dr. Andres De Los Reyes is an Associate Professor of Psychology and Director of Clinical Training at the University of Maryland at College Park. His publications have appeared in such journals as the Psychological Bulletin, Psychological Review, Psychological Assessment, and the Annual Review of Clinical Psychology. He is the Editor-in-Chief of the Journal of Clinical Child & Adolescent Psychology (JCCAP), and Founding Program Chair of JCCAP's Future Directions Forum.

Description

You have successfully mastered use of “story” when writing a manuscript, now how do you get it published? Publishing articles involves submitting scholarly manuscripts to peer-reviewed journals. A key component of publishing manuscripts involves receiving commentary about your work from peers in your field, and satisfactorily responding to such commentary. Yet, researchers rarely receive formal training on responding to peer review commentary. In this workshop, we describe evidence-based strategies for responding to peer review commentary, including strategies for how to compose cover letters for responding to such commentary.

Learning Objective

Identify evidence-based strategies for responding to commentary from peer-reviewed evaluations of your academic work.

Workshop Materials


Notes

Click "Expand" for notes
  • No one has taken a class on peer review, yet it is integral to operating in this field
  • Goal today: “make the covert overt” in the peer review process, appreciate the peer review process
  • Most folks have submitted manuscripts to journals and had them rejected and sent to different journals
  • Brief primer on the peer review process
    • Seven components:
      • Author chooses journal outlet for submission
      • Author prepares manuscript for submission format
      • Author submits manuscript and cover letter
      • Editor in Chief selects “Action Editor”
      • Action editor selects reviewers who evaluate the manuscript
      • Action editor reads manuscript and reviews , makes decision
      • If a revise and resubmit is selected, author revises based on reviewer commentary and sends back for further consideration (majority of time spent in this talk today)
  • Editor and Reviewer Selection
    • Editorial Board
      • Editor in Chief
      • Associate Editors
      • Editor Consultants
    • Editor in chief selects action editor
    • Action editor selects reviewers
  • Review Timelines
    • Most reviews take 2-3 months
      • Can take considerably longer if reviewers are hard to find
    • Medical journals: 1 month
    • Ranges depends on how hard it is to find reviewers, also from journal to journal and field to field
  • Review decisions
    • Action editor reads reviews and the manuscript
    • Decides to accept, reject, or request R&R (revise and resubmit)
    • Most papers start as R&R, so it is critical to have strong skills here- both in receiving and responding to commentary
  • Bad news about peer review
    • Not good reliability in peer-review process for papers, grants, etc.
      • Reliability= correlation between two independent assessors ratings of same submission
    • Example: Cicchetti (1991)[2] and Marsh et al.(2008)[3]
      • Cichetti resubmitted papers that were already accepted for review and reliability was still poor
      • Marsh determined that a respectable correlation requires triple the amount of reviewers
  • Good news
    • Once you are in R&R stage, peer review works well because it gives you ways to make your paper better (revised version of paper is always better due to constructive feedback)
  • Making peer review work for you- an anatomical map
    • Our Map- the board game operation
    • Nose- Do your detective work
      • Before the peer review process has started, you think about where to submit
      • You have some control over your reviewers- you can give advice about who would be a good person to review it. Maybe people who know the area or are known to be fair.
      • Reliability of peer review increases slightly when reviewers can be recommended
      • Some journals have a field in the submission process for this, but always include it in your cover letter either way
      • How to select reviewers (editor won’t use all of your reviewers)
        • In cover letter, identify 2-4 reviewers (template available in materials from Dr. De Los Reyes)
          • Respectfully request people and provide their emails
        • Make sure 1-2 are on the editorial board
          • Also include ad hoc reviewers who have content expertise
        • Editor is likely not to know newer researchers and their area, so it will be more helpful to provide suggestions
        • Not picking your friends, but people who have no conflicts of interests (fair evaluator)
        • Ask your advisors on who you should select
    • Limbic system
      • Accept your emotions
        • After you finally get your decision, you will be emotional and likely to make worse decisions. Give yourself time (2-3 days) before responding after you just waited months.
      • See illustration 2 for an example from Dr. De Los Reyes of a letter from peer reviewers
        • Harsh reviews- in this case, the reviewer suggested their team missed the boat on covering the relevant literature and doing what a literature review should do. The reviewer did not read the rest.
      • Wait to make edits to your paper after receiving your edits (peek and let it sit)
    • Frontal Lobe
      • Create a to do list first (itemization of the decision)
      • Template available in the materials section
        • Thank you for your feedback
        • We have highlighted in yellow all text revisions
        • Cover letter-Itemize comments word for word and then provide where you will make changes
    • Motor cortex
      • The cover letter becomes a plan for how to execute your actions
      • Address one comment a day- start with the easier ones
        • R&R should take a month or less if you have ~30 comments to address, which will give you plenty of time if you have a couple of months to complete it
      • Cover letter template available in the materials section
      • Thank the reviewer at the top for one nice thing they said to remind the editor that even the person that hated it said something nice
      • Also thank the reviewer for pointing something out
        • Never “punch back” at the reviewer
        • Make commentary constructive, not negative
        • There will always be hard reviewers. If they go low, we go high.
        • This makes the editors take the reviewers with a grain of salt
    • Arms
      • Embrace revision with “open arms”
      • Cover letter is very important
      • Fix everything
      • Most of the time, the reviewers are the same on the next submission
      • What if you can’t fix something?
        • Like a request to change theoretical framework or add analyses that can’t be done
        • Cite evidence to support your decision
        • Put it in the cover letter
        • Admit that you do not have data to make change, highlight as limitation
        • See illustration 5a-5c in materials section
          • Example 1- Paper was missing a chunk of rationale that they thought was necessary.
            • Highlight piece in paper
            • Beyond just adding it, put it in the cover letter for them to see
          • Example 2- Another problem the reviewers had was with the consideration of context in the paper. While they couldn’t put what was requested, they added it as a limitation in the discussion and suggest further context as a future direction
            • Reviewers are aware that no study is perfect
          • Example 3- Reviewers thought the paper needed to be streamlined. The writers removed some analyses and made the paper shorter
          • Example 4- the analysis was said to be low powered so the team added a power analysis
          • May want to have several subpoints in the cover letter of how you addressed a comment that seemed very important to the reviewers
    • Hands
      • It is okay to ask colleagues as well as co-authors for advice on revising manuscripts
      • Also fine to contact the editor but only for questions that really require their response. “Is X problem a dealbreaker” In other words, if you can’t revise something big, it’s okay to check in with the editor. Make sure to include what you said to the editor in the cover letter
    • Heart
      • Empathize with reviewers- they sacrifice their time to read your review
      • Make things easy for them
        • Highlight revisions in submitted manuscript
        • Note page numbers in cover page
        • If the comments become minimal, they may only rely on the cover letter and barely reference the manuscript
      • See illustrations 5d-5f in materials
        • Example 1- reviewer said that they should test if findings generalize from the whole sample to subgroups and even provided a useful citation and formula. The team wrote that they used the paper and formulas and what they got in the letter
        • Example 2- Added some requested analyses and an additional extra thing they might also care about to be on the safe side
    • Eyes- details
      • Look for typos! Editor’s think if the exterior is sloppy, the inside (the science) will be too. It colors the reviewers reading of the paper
      • However, this and a language barrier are very different things
    • Smaller points on the map
      • Gut
        • Peer review is a war of attrition- how long can you last? It is a quality control system
        • “ We would be pleased to make any further revisions”
        • Always want to tell the editor that you are here until the end
        • You will fix the paper until they accept. If you tell them, they’ll believe it
        • Get the benefit of the doubt by saying this
        • See illustration 6 in materials
          • Regardless of what happened, the paper is better!
          • “Whether the manuscript is accepted or not…”
      • Soul
        • Be a good academic citizen
        • Return the reviewer’s work by agreeing to review manuscripts
        • Accept review requests from journals where you publish
        • Email editors and let them know that you are available to review (Attach your CV to email) (once you publish 1-2 papers)
      • Wishbone
        • What if two reviewers disagree? Can address both with one thing. You get to pick- consider if you can tell whether one seems important to a reviewer
        • “Reviewers X said this and Reviewer Y said this. After much thought we decided to… (whatever you would rather do)”
        • Acknowledge conflict to editor, they will come back to you and suggest which way to go if you went the wrong way
      • Kidney
        • If you think you’ll be late, ask editor for more time!
        • Editor would rather have something that’s your best than a rush job. Most editors will give more time.
      • The Donkey-
        • Mean Reviewer
        • No one takes a reviewer class
        • Some reviewers have a bad day
        • If it’s nasty, let the editor know (do this rarely)
        • This is useful to both you and the editor
        • See illustration #7 in materials
          • Example 1- after weird tone from reviewer, before sent revision, sent email to editor explaining that you wanted them to be aware
            • The editor will often ask if you’d like to restart the process- go with “no.” You already have the R&R
            • Remember to be courteous
      • Even worse reviewers: the frenemy
        • The reviewer above may have had a bad day- these next people are worse
        • Reviewers leave fingerprints (sometimes you can tell who they are)
        • Some reviewers have a personal vendetta against you
        • Some comments give a reviewer's identity away
          • They tell you to cite themselves more
          • They don’t like your theoretical frame because it’s not the one they use
          • They don’t like your measures because it’s not the measure they use
        • What do you do?
          • Just need a reasonable suspicion
          • Happens to many people
          • 3 things- can be seen in the example in the materials
            • Name person in cover letter (don’t name more than two people not to review)
            • Respectfully ask that they not review the paper
            • Decline if you are offered a chance to review their paper (mention that you do this in the letter)
            • Cite philosophical differences
            • Take some of the blame
      • Broken heart-
        • Revise a paper and the editor rejects it…
        • Give it a few days
        • Ask if the paper is worth fighting for- new journal or revise substantially and send again?
        • A “fight” could reflect poorly on you- pick your battles
        • Moving on-
          • Read the reviews and see if there were deal breaking comments you couldn’t address
          • Should you shoot for a “lower tier journal”
          • Do not give up unless there is “extraordinary evidence”
      • Motivational “Fire”
        • Ulysses by Lord Tennyson as an example
        • The system is not perfect but neither is science. Science is hard, full of obstacles, slow, and underfunded.
          • Best parts of peer review strengthens papers by helping the writers and reviewers become better scientists
        • Part of a team of scientists in peer review
        • System cannot get the best of you because you are strong- persist and proceed
        • Goal of workshops is to give you tools. Push harder.

Questions from Audience Members

  • On a recent R&R, everything was addressed, but a co-author added comments about adding things in the mindset that a R&R is a chance to add stuff. However, this could be seen badly by adding things not suggested by reviewers.
    • Judgment call- potential to raise more questions than it answers. Could have different reviewers (editor will usually let you know if this is their plan)
    • Generally, add the thing you want to add. It may sink at that journal, but the paper will be better, which is the goal. Even if it has to go somewhere else.
  • Is there a difference in R&R and rejection with encouragement to resubmit?
    • Some journals treat them the same
    • Sometimes, it is rejected if the comments from reviewers would be very hard to incorporate.
    • Other times, they may see it as valuable to resubmit for different reviewers to look at.
  • Is it okay to email the editor to clarify about a comment? YES
  • Is there peer review within the peer review?
  • Does the editor look at the comments?
    • They should. This should precede their decision, however, the editor’s opinion can supercede the readers
    • Most folks have a philosophy of if you’re going to let someone down, do it as soon as possible.
    • JCCAP does not generally want to ask for, say, 3 revisions and then reject it. This is bad for a journal’s reputation
  • When should I decide to go to another journal after receiving an R&R?
    • Try to address their concerns to make your paper better
    • If you want your paper published sooner, then take it somewhere else
    • The worst they can do is say no
    • If you want to be published in that paper, then do the R&R

Job Search and Negotiation

Dr. Deborah A.G. Drabick, Ph.D.

Dr. Deborah Drabick is an Associate Professor of Psychology at Temple University. Her expertise is broadly in developmental psychopathology, and more specifically in youth externalizing problems. Her work includes such areas as risk and resilience, co-occurring psychological conditions, contextual influences, and intervention. Dr. Drabick has been funded by the National Institute of Mental Health, American Psychological Foundation, PA Department of Health, and Temple University. She currently serves as an Associate Editor for the Journal of Clinical Child & Adolescent Psychology.

Dr. Tara Peris, Ph.D.

Dr. Tara Peris is an Associate Professor of Psychiatry and Biobehavioral Sciences at the UCLA Semel Institute, where she serves as Program Director of the UCLA ABC Partial Hospitalization Program. Her research focuses on developing strategies for optimizing treatment outcome for difficult-to-treat cases of anxiety, OCD, and related conditions. She is the recipient of a career development award from the National Institute of Mental Health, a NARSAD Young Investigator Award, and awards from the Obsessive Compulsive Foundation, the Trichotillomania Learning Center, and the Friends of the Semel Institute.

Description

Do you plan to “go on the market” soon? Where is the best place online to search for job openings? How do you write your cover letter? Where do you find resources to assist you in preparing for the job interview? And when you get that job offer, how do you advocate for yourself and negotiate the right salary, benefits, and/or lab startup package? We leverage our extensive experience on “both sides” of job searches and negotiations to provide attendees with winning strategies for working their way through this multifaceted process.

Workshop Materials
Notes
Click "Expand" for notes

Overview

  • Where to look
  • What to expect
  • What to ask for
  • How to approach negotiating

Where are jobs listed

  • APA Psychology careers
  • ABCT, ADAA, SCCAP, D12 listservs
  • Word of mouth, mentor guidance for jobs not yet posted
    • Start sharing that you are interested as soon as possible
    • Let people know you’re looking so they can start sending opportunities your way

Overview of the Process

  • Variable across institutions
  • Start with a pool of applicants and narrow it down
  • Skype/zoom interview → in-person interview/job-talk
    • Does this person seem reasonable? Want individuals to fit into the context
    • In-person interview will typically last two days where you meet with faculty in order to assess your “fit” and they may compare their impressions of you in order to make sure they’re receiving the same reaction from you across the board
    • Try role playing these interviews with friends and faculty
      • Get a feel for answering questions and maintaining authenticity
      • Be prepared / process oriented
      • Showcase your accomplishments without being arrogant, but you’re not here to apologize for yourself
        • Speak with clarity and know yourself
    • How to present yourself
      • This is what I can bring to the table - said in a way that is not arrogant
        • Lines of inquiry
        • Here’s what im doing
        • Here’s what I hope to pursue
        • Where is your expertise and could you teach on it
          • “I TA’d for this, I independently taught this, teaching statement”
          • Show flexibility in what you could teach
        • Showcase your enthusiasm and your curiosity
          • Ask them what they might be looking for
  • Search committee convenes, reaches out to references
  • Get the offer
  • Second visit to look at housing, negotiate further
  • Final offer letter

Cover Letters & Statements

  • What makes you stand out
    • Pick one important thing from your CV to highlight
    • Avoid grateful language - you earned your accomplishments
    • Demonstrate interested and engage the researchers
    • Research statement
      • 2-3 pages
      • More focused on lines of inquiry, stating what you have done and what the implications are
      • “The research has generally shown X, and I’m planning on doing Y with it
      • What they want to see
        • History of funding
        • History of publication
        • Plans for the future
    • Cover letter
      • 1 to 1 ½ pages
      • Biggest question is: Are you fundable?
      • They want to see that there is some evidence for you to be fine in the field
      • To some degree, you will have to cover part of your salary through grants
    • Teaching statement
      • 1-2 pages
      • More like a narrative
      • In graduate school I did this, and then in post-doc I did this
      • Tip: look at the course catalogue for the institution so that you can directly cite what courses you would be able to teach and express your enthusiasm
      • “I’m really excited about this,” “I love mentoring,” “I’ve had experience mentoring with _____”
      • The teaching statement can easily become full of platitudes
        • Avoid that by making it a narrative
        • Reveal what your philosophy is and make it cohesive and thoughtful
        • What are the ways I illustrate what I am interested in
          • Presenting, Publishing, Writing Grants?
    • Job-Talk
      • People will be watching how you present
      • Can they see you in front of a class? Are you engaging
      • You want to be comfortable
        • Practice interviewing
        • Become acquainted with yourself and your strengths
        • They want to be able to picture you in the role, so make it as easy as possible for them to do that (ie., if you’re comfortable speaking, they can see you teaching their undergrads)
    • Service
      • Understand the expectations
      • Different jobs will require different levels/amounts of service
        • Being on committees, etc.
  • How does your experience map onto the job posting?
  • Past and future plans
  • Cohesive story with line(s) of programmatic work
  • Teaching philosophy

What Can I Ask for?

  • Salary
    • It’s okay to ask where people typically start
    • Information available online usually
    • Time to not be shy
      • Important to have a sense of how this works and what people make
      • Don’t be shy in what you want and also asking people for guidance
        • Ask people fresh in the job market about their situations/start-up packages
    • Make sure that you’re talking to people so you know what the going rates are
    • Where you start is incredibly important - your merit is based on your start
      • As you go up, where you start becomes more and more important
      • The worst thing they can say is no
    • “I’d like to talk about the compensation package”
      • Talk to recruiters, to your professors
    • Press harder
    • Do your homework so that you can ask for something that is reachable and in range
  • Title (years on the clock - choose when you next go up)
  • Space
    • The post-docs, the setting
    • Think through exactly what you need for
      • Running participants
      • Housing research assistants
      • Housing yourself
  • Start-up Funds: What do you need to cover staff, equipment, software, pilot research projects?
    • Justifying your cost is key
  • More time before moving (sometimes)
  • Protected time
  • Housing, parking, childcare
    • What is an appropriate way of addressing that your partner would be moving with you?
      • If you are interviewing somewhere that you know would work for you, emphasize your ability to relocate there
    • Spousal hire
      • Addressed in cover letter
        • “I’m jointly applying with”
      • Varies greatly
        • Sometimes partners get hired at different times
        • Sometimes partners are hired to different positions
      • Make it known upfront/in advance
        • If you know that both of you will be travelling together and need faculty positions, make sure it is known ahead of time
        • Making it known that the area works for you/your family helps the employer feel more comfortable in giving you an offer without the risk of you saying no
    • Childcare
      • Asking for resources that would help with things such as childcare would be more of an item to bring up with HR rather than the department
      • Things such as childcare may be on-site but you want to know if there’s priority given for seniority, if there’s a waitlist, etc.
    • Housing
      • Is there subsidised family housing
      • Is it
    • Parking
      • Ease of access, proximity to campus

Okay, but what do I say?

  • Helpful to refer to your mentor for advice on this
  • Always begin with gratitude:
    • I’m thrilled to have received this offer…”
  • And then:
    • “I’d like to clarify a few aspects of the compensation package…”
    • “Could you help me understand…?
  • And then:
    • “This has been really helpful. You’ve given me a lot to think about.”
  • Be mindful that if you take too long the department could lose the line of funding for your position
    • You have time to make your decision, but don’t go dark
    • Check in to get more information as is needed, but take your time

General Tips

  • Be transparent. Schools know that you may be looking elsewhere and it’s okay to say you need time
    • Know your deal breakers
  • Approach negotiation with a collaborative stance and be geared toward problem solving. Explore a few different paths to getting what you want. This way, you’re creatively looking for ways to make it work together
  • Send thank you notes to everyone. Personalize them where possible
  • Think about what you need to do th1e work you want to do
  • Trust your gut.

Preparing a Grant Post-Ph.D

Dr. Joshua Langberg, Ph.D.

Dr. Joshua Langberg is an Associate Professor of Psychology at Virginia Commonwealth University, where he directs the Promoting Adolescent School Success (P.A.S.S.) research group. His research focuses on improving the behavioral and academic functioning of children, adolescents, and emerging adults with Attention-Deficit/Hyperactivity Disorder (ADHD) and on disseminating evidence-based interventions for youth with ADHD into community settings. He has received over $12 million in funding from the Institute of Education Sciences, National Institutes of Health, and Virginia Foundation for Healthy Youth, and currently serves as Associate Editor for the Journal of Abnormal Child Psychology.

Dr. Susan White, Ph.D.

Dr. Susan White is Professor and Doddridge Saxon Chair in Clinical Psychology at the University of Alabama. Her clinical and research interests include development and evaluation of psychosocial treatments that target transdiagnostic processes underlying psychopathology. She is associate editor for the Journal of Clinical Child and Adolescent Psychology and the Journal of Autism and Developmental Disorders, and she is the Editor in Chief of the ABCT Series on Implementation of Clinical Approaches. Her research has been funded by the National Institutes of Health and the Department of Defense. She received her PhD from Florida State University.

Description

Submitting your first grant as a Ph.D. can appear on the surface to be a daunting task, with many expectations, requirements, and complicated forms. In this workshop, we leverage years of experience with extramural funding to explain the grant submission process, and provide attendees with concrete tools for submitting successful applications via multiple post-Ph.D. mechanisms, including project grants and K Series applications.

Workshop Materials
Notes
Click "Expand" for notes
  • Training/Early Career Awards is the Government’s attempt to get professionals established in their career sooner through federal funding
  • If you want to do a specific type of science with specific training and expertise you should do an R-series
  • K01: 6 years after post-Ph.D,, most common and a mentored research scientist, you have a faculty position
    • K01 has the broadest reach across institutions
    • K01 Considerations for grant is just as much about reviewing the candidate as it is about the research
    • K’s are awarded to institutions more so than the individual
    • Do not wait too long
    • Need something more to push the envelope in training
    • Panel of people 5-6 experts to help you achieve your goals in the training front: One in ethics, bio-stats, writing etc.
      • You need heavy hitters as mentors who have Post-doc training experience
      • Tangible examples of what they will teach or give you whether it be access to data, teaching new stat methods, or suggestions on ethics
      • Pick your Team to Fill in the Gaps
        • For instance an expert on a construct you are exploring but do not have that expertise
        • You need a senior person
    • You need to have depth not breadth
    • Include a list in a didactic fashion of training you have or will get for this grant
    • Publishing is important so they have confidence you can fly solo but there is not a hard number of publishing that needs to be completed.
    • Make sure what you are proposing is obtainable in the time you have and money (usually 4 years). You can always apply for an R-series later
    • One writing part of the K is like a mini-R series
    • Most K applicants are in medical settings since they are not usually in a position to having to teach or become a tenure professor
    • It takes so much time and it is one of the hardest applications to write
    • Before applying for a K grant having pilot data that lends credence to what your research would look like whether it comes from your lab or someone on your panel... Does not need to be your own always
    • K makes you a better candidate for tenure professor
      • It will make you more independent
      • A K will give you preliminary Data/Track Record to launch your career
      • A K-award is a stepping stone in your work
    • The aims should focus on research not your career development

What is a Track Record:

    • Do not need to be an expert in everything
    • Collaboration is viewed as a positive and a given
    • You need a biosketch of your contributions to Science: Paragraph + 4 pubs under each (want something to put there)
    • It is more important to have first author publications than 3rd or 4th so 2 publications in first author looks better than 5 3rd or 4th author publications
    • Put in the first part of section A about why you took time off or if you have any gaps in your career so they have context while reviewing the whole application
  • K23: “The Kangaroo”, Mentored patient oriented research career development award, often in medical school. Start as a I”m Post-Doc
  • F-Series are Fellowships for individual people, like graduate students or post-docs
  • There is a better chance of success for career development awards (32%) than K mechanisms
Picking Topics:
  • 2 main areas of focus called lines of research:
    • Your main area you have been in like your dissertation or your core and then a new area you are interested in and want to tie in like big trends such as the RDoC or High level statistics to develop it better and to cross over and become relevant. This is imperative for a K award

Block III Workshops (1:00 pm-2:15 pm)

Strategies for Developing a Research Program

Dr. Andres De Los Reyes, Ph.D.

Dr. Andres De Los Reyes is an Associate Professor of Psychology and Director of Clinical Training at the University of Maryland at College Park. His publications have appeared in such journals as the Psychological Bulletin, Psychological Review, Psychological Assessment, and the Annual Review of Clinical Psychology. He is the Editor-in-Chief of the Journal of Clinical Child & Adolescent Psychology (JCCAP), and Founding Program Chair of JCCAP's Future Directions Forum.

Description

Our first two writing workshops dealt with applying narrative tools to academic work and responding to peer review commentary, with the key goal of publishing a single journal article. How might you use these tools to connect separate articles together into a larger story? In research, our larger stories are the “research programs” we build from years of work and multiple articles. These are the stories we take with us “on the road” when interviewing for jobs and applying for grants. In this workshop, we discover how narrative devices commonly used in filmmaking actually help us weave related but distinct articles together into the “story” of an entire body of work.

Learning Objective

Describe strategies for using narrative tools to develop and describe your research program.

Workshop Materials
Notes
Click "Expand" for notes
  • How film-making informs telling your story
  • Where did this workshop come from?
  • Mentorship (How a hero’s welcome mirrors your own)
  • Two potential filmmaking tools
  • Questions
  • This workshop come from the students
    • An anonymous comment in the 2018 survey suggested this ‘creating a research program’ workshop
  • When should I “start” my research program
    • You already did! The previous workshop on storytelling (ABT) can be scaled up to research programs
    • Narrative tools are powerful
      • Cf. Houston, we have a narrative
      • Effective storytelling is very important in the research context
  • How a hero’s welcome mirror’s your own
    • Picture the moment you met your research mentor
    • Peter Parker and Tony Stark as an example
      • Many parallels with grad school
        • Imposter syndrome, reading a CV, reciprocal interaction, etc.
      • Note: in this film, Peter Parker doesn’t get his own background story
        • He only gets someone else’s story in which to tell his own
        • He gets to use Stark’s equipment and learn from him
        • Note that mentors grow through getting to know their students as well
    • Our Research Programs: how we tell our story
      • Our mission: write and tell our own story, extend our mentor’s story
  • How to use filmmaking tools to tell your story
    • The challenge (part 1): rather than being a rogue planet, your mentor’s lab exists in a larger universe and has interconnected labs focused on related research topics
      • If mentor’s lab is interconnected, it cannot quickly change course
        • The lab has influence on you research program
          • This may feel stifling
      • However, this focuses your attention by
        • Providing set of theories, methods, etc
        • Helping you to tell your story
      • In an ideal world, you story pivots off of your mentor
      • During the workshop, we will overview how Dr. De Los Reyes tells his story using his graduate student, Bridget Makol’s, work
  • Tool #1- Shared Universe
    • Finding your passion
      • Find your motivating question for research that underlies all of your work
        • How you open your talk about yourself
      • Firstly, a description of Dr. De Los Reyes’s lab
        • It can be helpful to take the beginning of a talk to frame the work
          • e.g., “there are many questions I’m interested in, but if I had to boil it down, it would be this:”
        • Why different people with the same behaviors are evaluated in different ways
        • Oftentimes, different people are asked the same questions about a client
        • Each may give a different answer
        • What sort of care will we give and is it working?
        • Children and adolescents lead complex lives. This may lead to different evaluations from different raters
        • Example- A person with social anxiety who can’t speak in large groups but are fine one on one, and other clients may be the reverse. Another client may be fine unless the setting is a social setting (adolescents differ from each other, yet they all may have social anxiety)
        • Adolescent will behave differently based on their surroundings like snowflakes (they are unique) rather than a cookie cutter mold
        • Take away your discipline and describe your question so that anyone can talk to you about it
      • Activity- You try coming up with a “burning question” that you’re studying
        • Like- how does language develop and what happens when it doesn’t
        • One valuable piece of this is being able to communicate your idea(s) to people outside your discipline
    • The mentor’s lab is like a galaxy
      • The center is the burning question- the big picture
      • Stars are individual questions
      • Planets are pieces of work that you think are good examples of turning your question into science
    • Concept of the shared universe-
      • Like the marvel movies- the individual movies tell their own standalone stories- like planets in a galaxy of films
      • The stories still fit together and connect (larger story= star)
    • Your theoretical framework
      • Need more than a question and stories
      • Your Thanos- how your star connects to all the planets- the purpose behind everything binding it
      • Typically appears throughout your scholarship for coherence
      • Theoretical framework, generally, is how you answer your burning question and then the studies test this suggestion
        • Your own set of ideas
        • Your mentor’s ideas/ lab framework
        • Well-known framework from another lab
    • Clarifying Questions
      • Can you explain the difference between stars and planets?
        • Star= question
        • Planet=studies
  • The Challenge- Part II
    • The mission is to build a solar system for your work- more than one study or project
      • If you only have 1 planet, there’s little evidence you can synthesize work (and also build a research program)
      • Your solar system should highlight your ability to connect ideas
    • How to do this with only a few years?
      • Mentors probably have had decades
    • Tool 2: Trilogy
      • The tool of the trilogy
      • Example of Wes Craven who has a background in psychology
      • He understood that when you see a horror film you aren’t just a witness, you’re interacting with it in a safe environment
      • He knows how to tell a good story and connect with an audience- exactly what you want to project when you talk about your research program
      • You want to make it easy for people to connect with you and your work by making it easy to connect with you and your work by making it connected
      • Planets (papers) in your solar system are like three movies in a trilogy
      • Plan out each planet’s role
        • Craven’s “Trilogy Guidelines”
          • Make people care- The first “movie” has to excite people and get them to come back for more
          • Introduce conflict
          • Surprise them
        • Guideline 1: Make people care (Deep character development)
          • Audience commits to character’s well-being
          • Bad news throws off audience and makes them care about other characters
          • First paper should get people excited, not necessarily be perfect
          • Movie 1 (aka your first project)- excitement, pride, get the audience to care
          • Bridget’s movie 1- The take home message is that there is a history of discrepancies in reports of youth mental health
            • They found predictable patterns of where the discrepancies appear. These patterns tell us who is most likely to have informants disagree
            • Tools allow assessors to link patterns in depression too
            • Set up movie 2- has to be something else you can do- an improvement on film 1. Taking your story to new heights and introduce conflict.
        • Guideline 2: Introduce a conflict
          • You have the audience’s attention.
          • Like movie 1 with new nuance (different sample or setting)
          • Set up a conflict in move 2 to resolve in movie 3
          • Bridget’s movie 2- used inpatient samples, also looked at discrepancies between parents’ and children’s reports of depression
            • Found that patterns in assessments predicted treatment characteristics during the stay (solved movie 1’s conflict because same paradigm works in this population too)
        • Guideline 3: Surprise them
          • Movie 3- resolves movie 2’s twist, shows something that is a new truth, take audience back to beginning of story
          • This creates the circumstances for the third movie to solve
          • Bridget’s movie 3-
            • If we have know about discrepancies, why are we letting them come to us? Why not control who we have as informants.
            • A GPS may disrupted by overhead objects as overhead satellites are blocked and you can’t be triangulated as well
            • If we want assessments to enhance prediction- we need to find sources in different points in space to “triangulate” the symptoms (choose informants who will give different information on purpose in order to help predict outcomes)
            • Pick information sources that you’re confident in with different perspectives. Extract variance across three informants.
            • This technique is powerful- incremental prediction of observed behavior is enhanced
          • Epilogue-
            • Trilogies in science are different than movies
            • Must end with resolution to conflict
            • Must know that research is not over
            • Example with Bridget- now she can use this information to work with individual clients, in clinical work, and with a different set of informants
            • “I know this is a limitation and this is how I want to tackle it”
          • Element of surprise
            • When it comes to preparing a research program, your preparation is done post-hoc, retrospectively
            • It is normal not to know how to connect your research until you need to (you are not alone in not knowing how to connect your research)
            • Reminder: during training, no one has complete control over scholarship
              • At mercy of mentor’s resources and larger program
              • No one has this figured out ahead of time
            • Activity 3- What’s Your Trilogy?
              • Have the audience explain their trilogy
              • Example 1- Why do some kids struggle with making meaningful social bonds? How can we help them make more meaningful friendships?
                • Movie 1- show differences in social functioning among those with ASD
                  • How do these factors lead to differing neural development?
                • Movie 2- conducted meta-analysis demonstrating abnormal responses to faces, showing processing differences
                  • Then how do you help these children?
                • Movie 3- social skills interventions, showed efficacy of an intervention for those with ASD
                  • Future directions: mediators, moderators, active ingredients, how patterns of neural conductivity reflect ability to respond to different experiences (neural plasticity)
              • Example 2- how young children exposed to stress manifests biologically, how parental influence changes this
          • Questions
            • Do the papers have to be chronological? NO
              • Generally, the first movie is the first thing you did
            • Can a review paper be a “planet”? YES
              • Meta-analysis can be great! You uncover patterns here
              • Principle should extend to any discipline (e.g. humanities with books)
            • What about publications outside of your narrative?
              • This may come up outside of your job talk
                • Say that you will answer these questions outside of your talk
                • Faculty looking for synthesis, not everything you’ve ever done
                • Don’t fall into the trap of running out of time, you want to tell people what you want to do next
            • What about aspects important to your identity, but outside of research? Do you include these in your job talk?
              • Find a way to connect it to your program
              • Highlight them when talking one on one with faculty
            • Can the trilogy method be applied to more than three pieces of work? MAYBE
              • Don’t try to jam everything in
              • If you are giving a job talk, you’ve already been screened, you probably only need a trilogy
            • What if you pivoted in interests in your life?
              • Leave it out of your job talk/narrative
              • Give a good story, answer questions if you need it
              • When you think about stories in this way (burning questions that are broad), it’s easier to work together with other scientists

Networking at Conferences

Dr. Deborah A.G. Drabick, Ph.D.

Dr. Deborah Drabick is an Associate Professor of Psychology at Temple University. Her expertise is broadly in developmental psychopathology, and more specifically in youth externalizing problems. Her work includes such areas as risk and resilience, co-occurring psychological conditions, contextual influences, and intervention. Dr. Drabick has been funded by the National Institute of Mental Health, American Psychological Foundation, PA Department of Health, and Temple University. She currently serves as an Associate Editor for the Journal of Clinical Child & Adolescent Psychology.

Dr. Matthew Lerner, Ph.D.

Dr. Matthew Lerner is an Assistant Professor of Psychology, Psychiatry, & Pediatrics in the Psychology Department at Stony Brook University, where he directs the Stony Brook Social Competence and Treatment Lab. His research focuses on understanding mechanisms of and developing interventions for social and emotional functioning (in particular peer relations) among children and adolescents with autism spectrum disorders and ADHD. Dr. Lerner has received over $8 million in funding for his work from the National Institute of Mental Health, Brain & Behavior Research Foundation, American Psychological Foundation, Simons Foundation, Alan Alda Fund for Communication, Arts Connection, and Pershing Charitable Trust.

Description

To an early career scientist, attending professional meetings can be an overwhelming experience, with many opportunities to not only learn new things but also connect with like-minded scholars in the field. In this workshop, we demystify the process of networking at conferences, and provide attendees with concrete tools for developing and maintaining professional relationships with conference attendees.

Workshop Materials
Notes
Click "Expand" for notes
  • Outline
    • Why is networking important?
    • How do you do it?
    • What if you’re really really social?
    • What if you’re not really really social?
    • How to maximize opportunities.
  • Why is networking important?
    • Psychological science is a Team Science and the way that is built is by creating networks for communication/collaboration through networking
    • Important for connecting one another to jobs that they may not have been connected to otherwise
    • Networking is more important than grades in predicting your success coming out of grad school
  • Why it is important: Things you might not know
    • Professors want to connect with less senior individuals
    • It’s important to ask yourself “Why do I want to do this?”
      • Forging interpersonal connections, exchanging ideas, thinking about collaborations
        • It’s more important to have positive interactions than strategizing as to how you can make the person like you
        • Part of building your reputation is being likable
        • It’s not good to network solely to jumpstart your career
    • Types of conversations that professors might like or find engaging
      • Conversation that reflects curiosity, enthusiasm, excitement, interest, etc.
      • Conversation that references the professor’s work and shows admiration, respect, an attempt at understanding and making a personal connection
    • How to make yourself stand out
      • Being someone that generates/inspires a positive feeling in another person
      • Showing familiarity with a person’s work and asking thoughtful questions
        • Think about who’s one person you want to meet and one thing you want to talk about before a conference
      • Have a brief summary of your research interests prepared
        • Have an elevator pitch - or at least a rough outline
          • Be able to get to the point quickly because your time is often very limited at a conference
        • I’m studying X with Y and I’m planning on doing _____
  • How do you do it?
    • 80/20 rule
      • 80% networking and 20% content
      • You get the most out of a conference by networking
        • Even if you’re signed up for a talk, it’s okay to skip something content based to talk to someone that you could have valuable conversations with
    • Where to find the people you want to network with
      • Everywhere in and around the conference process (ie., airport, hotel, lobby, coffee shop)
      • Sometimes even during talks and poster presentations
      • It’s important to read the person and think about timing
        • Sometimes just saying “Hi I’m ___, I’m applying to you for grad school in the fall, is it okay if I email you about a few questions,” is better than having a 10-minute conversation
        • Being brief and concise is paramount for sparking connections at conferences with potential collaborators, employers, and/or mentors
    • Be genuine
      • Think about what you really want to talk about
        • Research topics, methodological questions, something you find interesting that you think they will as well
      • “Exposure approach”
        • Work your way up from people you find less to more intimidating in order to become used to networking
        • Break down the myth in your head that these people are more than human while moving up your hierarchy
    • Be nice to EVERYONE!
      • The people who are going through the same experiences with you/are on the same level as you will one day be your colleagues
    • Be aware of how you present yourself
      • Be mindful of how you come off in interactions while not being artificial
        • In certain contexts you may have to dial it down
    • Use contact tools (ie. cards, paper, your phone) to follow-up
      • It can be helpful to have these during poster sessions
      • Helps make sure that your contact doesn’t end after your conversation
    • Don’t cling
      • Don’t completely cling to your advisor, but you can use them to start conversations
      • Especially don’t cling to someone you just met
      • Leave them wanting more
    • Present your work and be proud of it
      • Don’t underplay your work due to the power differential
        • If you were asked what you did, they genuinely want to know
    • Be generous
      • Be generous with your time and effort in helping others, it can be as little as giving the person next to you a pen
      • Being a good person is way more important than being perfect
    • Ask a fail-safe question
      • What are you working on now?
    • Ask about process issues
      • Methods, execution, difficulties, struggles, setbacks, and anything else that went into the research that isn’t conceptual - “How did you do it?”
    • Attend a presentation given by the individual of interest
    • Ask about current events in the discipline
    • Talk about the conference since it’s a shared experience
    • Ending the conversation
      • Don’t ask them for anything
        • Don’t ask someone to send you an article that has free access
        • Don’t put something on their plate
      • But do ask about how to remain connected
  • What if you’re really social, or really not social?
    • There are risks of being too extroverted at a conference
      • Don’t be overbearing
      • You could also miss opportunities due to not being present in your current interactions
    • If you’re too introverted, be mindful that some people are nourished by interaction and some are exhausted by it and that’s okay
      • Think about what parts of the conference can you get something from, and that’s okay for you
    • Exit strategies are just as valuable as entrance strategies
  • How to maximize opportunities
    • Attend a presentation by the professor or the professor’s student
    • Know the professor’s current research and/or an important paper that was influential to your thinking or own research trajectory
    • Consider the professor’s methods and how those could inform your own work
      • You can learn from them about it
    • Identify current controversies relevant to the professor’s work
    • It’s okay to be direct with your approach

Work-Life Balance

Dr. Joshua Langberg, Ph.D.

Dr. Joshua Langberg is an Associate Professor of Psychology at Virginia Commonwealth University, where he directs the Promoting Adolescent School Success (P.A.S.S.) research group. His research focuses on improving the behavioral and academic functioning of children, adolescents, and emerging adults with Attention-Deficit/Hyperactivity Disorder (ADHD) and on disseminating evidence-based interventions for youth with ADHD into community settings. He has received over $12 million in funding from the Institute of Education Sciences, National Institutes of Health, and Virginia Foundation for Healthy Youth, and currently serves as Associate Editor for the Journal of Abnormal Child Psychology.

Dr. Sarah Racz, Ph.D.

Dr. Sarah Racz is an Assistant Clinical Professor at the University of Maryland at College Park, where she serves as a Research Educator with The First-Year Innovation & Research Experience (FIRE) program. Her research seeks to understand both the proximal (e.g., parenting, family functioning) and distal (e.g., neighborhoods, schools) influences on child externalizing behaviors. Dr. Racz is also interested in the application of statistical models of change. Her research has appeared in such journals as the Journal of Consulting & Clinical Psychology, Psychological Assessment, Journal of Clinical Child & Adolescent Psychology, and Clinical Child & Family Psychology Review.

Description

Sometimes it feels like everyone in our field is “always on task” and unable to “unplug”. But is that a realistic view of how we balance our work lives with our lives outside of work? In this workshop, we discuss the competing demands placed on us across our various work, family, and social spheres; and strategies to manage these demands in the necessary pursuit of healthy, balanced lives.

Workshop Materials
Notes
Click "Expand" for notes
  • Overview and Caveats
    • Work-life balance does not mean that you are not incredibly busy and stressed from time to time
      • If that’s your goal, play the lottery
    • So what does work-life balance mean?
      • Ending the day/week feeling like your time and efforts were put towards your life’s greatest priorities
        • Note: nothing in here says “not stressed”
      • A feeling of satisfaction that your time was well spent
    • Caveat 1: strategies described here work well for some but maybe not others
    • Caveat 2: work smarter, not harder
    • Caveat 3: in graduate school, especially, no one really has the perfect work-life balance. It’s a very hard program, at the end of the day
  • Know thyself
    • The first, most important step in achieving a good work-life balance is honest, self-reflection and getting to know yourself better
  • Practice what you preach
    • As clinical psychologists, we teach others all the time how to cope and manage, but we don’t normally practice those very things in our own lives
  • Step 1: Operationalize your priorities
    • What does “I priorities my family” mean? What does “I prioritize my writing” mean?
    • Operationalize your priorities. Pretend you are creaitng a treatment plan.
      • Start broad with a list of priorities, then create specific, measureable, and feasible goals (*know thyself - don’t set yourself up to fail)
      • “I will be home by 5:30pm every day”
      • “I will be at the gym by 6:00pm every Mon/Wed/Fri”
      • “I will write for 1 hour on Tues/Thurs”
  • Newton’s Third Law: Every action has an opposite and equal reaction
    • Given how busy we are as students, fellows, faculty etc…, when we decide to change something, it almost always means more than one thing will have to change
      • We don’t want this to be true
      • However, pretending that we can just add things to our plate without taking away sets us up for failure
  • Step 2: Map your current patterns and schedules
    • If every action is going to cause a reaction, we have to start by knowing what we are doing and when
    • Start by mapping out your current daily schedule
    • Down to 30 minute increments
      • Ideally, including evening and weekends
    • Ex: Lawyers bill in 15 minute increments and thinking about our days the same way can help us be more efficient
  • Step 3: Making Conscious Choices
    • This is where self-reflection and being realistic and honest comes into play
    • Where are you spending your time? Be honest.
    • When do you work best? When are you most efficient? Do they overlap?
    • When are you working most and how are you using your time?
    • BE HONEST
    • This is a choice, make sure you are okay with that choice
  • Example:
    • “I want to prioritize my family”
    • What does that mean?
      • I want to be home by 5PM every evening to help with dinner and homework (defined goal)
    • Promotion/New Position in your career = not meeting goal
    • Making it work means an equal reaciton somewhere else
    • Most efficient with work in early morning, quiet, empty building
    • Choice: get up earlier and get to work earlier
      • Get to work by 6:30am
    • Reaction: need to go to bed by 9:30pm
    • At this point, let’s say you’re getting there but not quite there
    • So, let’s start packing lunch!
      • Normally, 30 minutes waiting in line for lunch
      • Now, you can eat while you work
    • Slowly, your goal is accomplished
  • Routine and Structure
    • It really helps to create a work-life balance
    • In graduate school, though, routine and schedule is very rare, which is why maintaining a healthy work-life balance is harder in graduate school
  • Set yourself up for success
    • Be present in your goal
    • If your goal is to be home by 5pm does that mean home by 5pm but still answering emails? Or home by 5pm and completely logged off?
  • Acceptance
    • Some people value doing things different every day and taking things as they come
    • Some people have life situations that limit their ability to follow a structured routine
    • This can create stress as it is challenging to plan and forces work/life choices to be made in the moment
      • Semi-conscious choices
      • Often a middle ground
    • Sometimes you just have to accept that you’re just barely keeping your head above water, and that’s okay
  • Having a Young Family While Working in Academia --Can you really have it all?
    • The Plan
      • Everyone has “a plan”. A plan to get married by a certain age, a plan to have kids by a certain age, a plan to graduate by a certain age, a plan to get a job by a certain time, etc…
      • But your “plan” rarely works out the way you think it will
    • Strategies for success
      • Be Flexible
        • Have a plan, but know that the unexpected will happen
        • Think outside the box
            • You don’t have to become a professor. You can be in private research firms, non-profit organizations, think tanks, etc…
        • Find your “split shift”
          • Be flexible about your work day; work hours are not just 9am-5pm
          • For some people, because of their lives during the day, have a “split shift” where they work from 10pm-1am
          • For others, they get up at 5am as their “split shift”
      • Be organized
        • Prep as much as you can for the next day the night before
        • Prioritize
        • Keep expectations reasonable and realistic
        • Calendars are you’re friend
          • Shared google calendar
        • Lists, post-its
        • Family management apps
      • Be a good communicator
        • Divide and conquer!
        • Work with your partner (if you have a partner) to set up a plan that works for you and your family
          • Full time or part time work?
          • Nanny? Daycare?
          • Who is picking up, dropping off, cooking dinner, taking kids to extracurriculars, picking up, etc…
        • Ask for help, and using your village
      • Be a superstar!
        • Put your best foot forward, especially when you first start your job
        • Supervisors are much more willing to be flexible and work with you if you have an established history of being trustworthy, hard-working, efficient, and effective
        • Work closely and communicate openly with your supervisor, and be honest about your needs
      • Be an advocate
        • Know your rights, and responsibilities as a working parents
        • Read and understand your institution’s policies regarding parental leave
          • How much time can you take? How much will you be paid? How is leave calculated? Can you take more leave, or take unpaid leave?
          • Will going on leave put a pause on your tenure clock? Is this automatic or do you need to ask?
          • After going back to work, are there policies in place for taking care of a sick kid? If breastfeeding, are there policies regarding pumping?
      • Be Present
        • Be extra productive during work hours
        • Put the phone/email/computer away for a few hours every day when you are with your family
        • Know thyself and accept your limits
          • What do you value? What is important to you?
        • Do something, even something small, that you enjoy every day
    • A fresh perspective
      • “Working Life” (Teahan, 2018)
        • You can’t separate work and life; there will never be a 50/50 balance
        • *Work smarter*
        • Find an employer that values what you value and can provide flexibility
          • But know that flexibility comes at a cost
        • Make small adjustments each day to fulfill all obligations
        • Recognize that all your “roles” contribute to your professional development
        • And, pay it forward

Ceremony for the Future Directions Launch Award (2:30 pm-3:30 pm)

Nicole Lorenzo

  • Award Winner in the area of XXXX
  • Ph.D. candidate at Florida International University
  • Presentation Slides
About the award winner
Notes
Click "Expand" for notes
  • Parenting: Helping Families Left Behind
    • Criminology Undergrad Degree
    • Parent Training is one of the most common types of treatment training
    • Families do not always get to a healthy position after PMT and then there are a lot of families that do not even make it into the clinic
    • Parent Training is not usually covered by health insurance and it costs money and time since it is 18+ weeks of training and so it is usually only available through research funded studies
    • How do we make it more accessible to all families?
    • Need to focus Emotional Regulation in Parent Training and treatment to help children succeed especially if we start early intervention

John L. Cooley

  • Award Winner in the area of XXXX
  • Received Ph.D. from the University of Kansas
  • Presentation Slides
About the award winner
Notes
Click "Expand" for notes
  • Adversity Within the Peer Context
    • Peer Group victimization - Universal Phenomenon
    • 24% experience chronic victimization from K-12th grade.
    • Some are affected by peer victimization more so than others  
    • Some children who are victimized will become increasingly aggressive over time
    • Maltreatment has been linked to peer victimization
    • He’ll be examining the risk factors that occur in daily experiences related to peer victimization through a daily diary study that will be starting in the fall

Erin Kang

About the award winner

Erin Kang is a graduate student in Clinical Psychology at Stony Brook University. She earned her B.A. in Psychology and in Molecular & Cell Biology at the University of California at Berkeley and M.A. in Counseling at Gordon-Conwell Theological Seminary. Her research interests include understanding various mechanisms and developmental processes underlying autism spectrum disorder (ASD) and evidence-based interventions for ASD. Her research advisor is Dr. Matthew Lerner.

Notes
Click "Expand" for notes
  • Understanding Plasticity in Social Processes to Improve Social Functioning in Youth
    • Why do kids struggle with making and maintaining social bonds?
    • Aims to gain a deeper understanding of deficits in social functioning and how they might respond differently to differing experiences.
    • Examining these effects over the course of development by using EEG measurements to better understand brain connectivity
  1. Olson, Randy (2015). Houston, We Have a Narrative. University of Chicago Press. ISBN 9780226270845.
  2. Cicchetti, Domenic V. (1991/03). "The reliability of peer review for manuscript and grant submissions: A cross-disciplinary investigation". Behavioral and Brain Sciences 14 (1): 119–135. doi:10.1017/S0140525X00065675. ISSN 1469-1825. https://www.cambridge.org/core/journals/behavioral-and-brain-sciences/article/reliability-of-peer-review-for-manuscript-and-grant-submissions-a-crossdisciplinary-investigation/D02B8D10E8F804E15B5FF882421B011E.
  3. "PsycNET". psycnet.apa.org. Retrieved 2019-07-02.
  4. 1 2 White, Susan; Langberg, Joshua (2019). "Preparing a Grant Post-Ph.D with Dr. White & Dr. Langberg". Open Science Framework. doi:10.17605/osf.io/gyaz4. https://osf.io/gyaz4/.
This article is issued from Wikiversity. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.