Grant Assistance from RDA: Preparations and Expectations

The bronze Jayhawk dusted with snow in late spring

Research Design and Analysis (RDA) scientists are well-equipped and ready to support investigators in all aspects of study planning. This may include:

  • translating research questions into specific aims
  • constructing experimental designs and sampling plans
  • determining appropriate sample sizes at each level of analysis
  • selecting measurement instruments
  • developing a comprehensive and tailored plan for statistical analysis

We also provide guidance on how advances in quantitative methods can be leveraged to make the most of existing or new data. Notably, the RDA scientists are skilled and experienced communicators who help investigators articulate all of these design and data analytic considerations in proposals to a variety of funding agencies (e.g., IES, NIH, NSF, NIJ). In order to help us help you prepare the most compelling proposal possible, we have provided the following guidelines for how we can best work together to achieve the best possible result. 

Broadly, you should expect an iterative process in preparing a grant application with the RDA scientists. Investigators frequently come to us with a strong sense of their proposed research design, but with lingering questions about some details. But rarely is it the case that we can provide concrete answers that will not meaningfully affect the rest of the proposal. Likewise, the proposed statistical analyses will likely require at least some moderation of the wording and ideas within the research questions and methodology in order to create a coherent and consistent document. Therefore, it is important to anticipate time for these types of back-and-forth conversations when planning your timeline, as elaborated below. 

An Ideal Suggested Timeline of Activities

To ensure that we can give your grant our maximum attention in order to create the most persuasive grant possible, we suggest the following timeline for our help in preparing your application. We note that the targets below are not set in stone, because our capacity to assist you will always depend on the size of our queue at any given time. For example, we may be able to accommodate shorter timelines for grants to be submitted during off-peak months, whereas we will need longer timelines for grants to be submitted during peak times (e.g., IES submission in August). 

10 weeks from submission

Ideally, initial conversations with the RDA about a proposal should occur at least 10 weeks prior to submission for regular grants (12 or more weeks during peak times). At this point we can help shape the design of the proposed research prior to generating an analysis plan.

5 weeks from submission

The next step, to be completed at least 5 weeks prior to submission, is for the investigator(s) to assemble ALL of the information requested in the section Necessary Input for Writing an Analysis Plan. Please keep in mind this very important step may require multiple back-and-forth conversations with the RDA and other collaborators in order to make and describe all of these decisions, and thus may take longer than initially anticipated. 

Once complete analysis plan input has been provided, it will take a minimum of 1–2 weeks for the RDA to write the first draft of the analysis plan. At this point we can also help create a reasonable, well-informed budget for our involvement (see Creating a Budget below). Upon receipt, the investigator should try to incorporate the analysis plan content into the proposal, monitoring for consistency in ideas and in wording (e.g., what constructs and measures are called).

1-2 weeks from submission

A near-complete draft of the entire research plan should then be submitted to the RDA for review at least 1–2 weeks prior to submission. We will provide feedback on the readability and consistency of the entire document, which can be helpful given our non-expert point of view for the non-analysis content. Time permitting, we can also provide preliminary feedback on the rest of the proposal prior to this final stage.

After reviewing the near-complete version of the entire research plan and our portion of the budget, we can then provide biosketches and a letter of support to reiterate our commitment to your project as needed to complete our portion of the application. 

Less-than-Ideal Timelines: Frequently Asked Questions

So what happens when we are presented with a less-than-ideal timeline for grant assistance? We will always try to assist investigators with such requests as much as we can, but our capacity to do so may be limited. In recognition of a less-than-ideal, investigators may try to reduce the scope of their requests for our assistance, such as to the levels illustrated by the questions below. However, our experience has taught us that even seemingly minor contributions can require a substantial investment of our time and effort, as elaborated below. 

Just as with any other co-investigator or collaborator, inclusion of such support materials functions as a tacit approval of the project and its proposed methodology. Please note: If we do not have the opportunity to review and contribute to your research plan, then we cannot provide personalized grant support (i.e., a letter of support endorsing your project or for any RDA member to have a percent effort in your budget). Instead, in these situations you can list and budget for future RDA contribution using a more general fee-for-services mechanism (see Creating a Budget, below).

Power analyses are always model-specific, so we cannot provide meaningful estimates of statistical power for a desired model parameter without a clear preliminary understanding of the model it would have come from, as well as the chance to provide input on the modeling process itself. 

More often than not, seemingly minor changes to the methodology will require large changes to the analysis plan. Further, RDA participation on a project means that we must stand behind the analysis plan and ensure its appropriateness for ourselves. For these reasons, proposal revisions are usually rendered new proposals in execution (and require a full timeline).

Creating a Budget

The RDA currently offers grant preparation assistance for proposals submitted through the LSI free of change. With respect to planning a budget for your proposed project, our philosophy is simple: if you need our help to write about it, then you likely will need our help to do it. It is important to create a budget that is reasonable and believable given the data management and analyses required. Support for most projects follows a U-shaped function over time, in which support needs are high when the project is getting started (e.g., for creating data entry forms, data storage systems, confirming measurement procedures), support needs are lower while the data are being collected, and then support needs increase once data analysis and dissemination begin. 

There are two main avenues for budgeting for RDA support. The first option is to include some percentage of time for a named individual. This option works well if you are anticipating a large number of hours to be required by a person with a particular skill set. The second, more general option is to include fee-for-service hours, which has the advantage of being fulfilled by multiple individuals in flexible time intervals.

The RDA is available to assist with more general statistical analysis (i.e., those not in conjunction with proposal preparation) at hourly rates. We are always happy to meet with investigators and to offer our input as to how any analysis needs can best be met.

Necessary Input for Writing an Analysis Plan

In order for us to write a customized, informed, and compelling plan for the statistical analysis of your proposed data, we first need to know the information listed below. We have provided this list as a guide for investigators, but what follows may not be sufficient for all projects.

Funding Source. We need to know the funding agency, the mechanism, the length of the project, and the length of the research plan. It will also be helpful if you can provide a copy of the RFA/RFP you are answering when applicable.

Research Questions (e.g., the NIH Specific Aims). The wording may be further revised, but we need to have a clear understanding of the specific questions to be answered. 

Sample. We need to know the following about the source of your proposed data, even if you are unsure of the sample sizes needed.

  • Who will provide the data? Will a single source or multiple sources be used (e.g., data from children only, versus data from children, their parents, and their teachers)?
  • Will participants be assessed on more than one occasion? If so, how many occasions, and how far apart in time will each occasion be?
  • Will your participants be assigned to new groups (e.g., control versus treatment)? How many groups, and how are they differentiated?
  • Are your participants part of any intact groups (e.g., families, classes)? Even if this grouping is not of interest, we will most likely need to account for it in the analyses.

Measures. For every construct mentioned in your research questions, we need to know exactly how it is to be measured by observed variables. For example:

  • Is it a scale? How many subscales? How many items? What is the item response format(s)? Is it a well-known scale with previously-reported psychometric properties (e.g., reliability, model fit) or something new that will need to be evaluated per se?
  • Is it a behavioral task? What is the task like? How many blocks of trials/items, and how many trials/items within each block? What outcomes does the task provide (e.g., response time, accuracy, d-prime), and how do they indicate the construct?
  • Is it a biological measure (e.g., cortisol, heart rate)? How do the obtained values represent the construct?
  • For studies with multiple sources of data (e.g., data from children, their parents, and their teachers), who will provide each of the measures?
  • For longitudinal studies, at how many occasions will each measure be given?
  • For each measure, have you used it in your own research before? 
  • For each measure, how is it distributed (i.e., how does it vary across participants)? For example, is it likely to have a floor or ceiling effect? Does it vary continuously otherwise? Is there likely to be more than one peak?

Roles of Variable. We need to know how the outcome variables from each of these measure will function in a model. For example:

  • Which variables will only be predictors? These might include demographic variables, covariates, or group assignments (e.g., control versus treatment).
  • Which variables will only be outcomes? 
  • Which variables could be predictors or outcomes (i.e., depending on the question)?

Pilot Data or other Sources of Expectations of Results. We need to understand any empirical basis for anticipating results and possible problems to anticipate. For example:

  • Do you have pilot data that needs to analyzed in order to be included in the proposal? Note that if a new style of analysis is planned that would be possible to do with the existing pilot data, it may be useful to do so for the proposal.
  • Are any of the above measures completely new to your own research? 
  • For variables measured at multiple occasions, do you have a sense of how much they actually change over time?
  • Do you have pilot data or previous literature that can be used to estimate effect sizes with which to conduct power or sensitivity analyses?