Sandia Fellow Kathy Simonson has read hundreds of research proposals during her career.
She’s also served on several investment area teams, which review and score Laboratory Directed Research and Development ideas and proposals. She noticed that among the weaker proposals, many of the same issues kept cropping up, including ideas that were not adequately structured within the context of the LDRD call or proposals that were poorly written and didn’t address the key question “Why should anyone care?”
“Even folks with tremendous educational backgrounds did not always understand how to develop a good research proposal,” she said.
This experience, combined with the observation that her own center had largely “self-selected” out of the LDRD program, gave her the idea for a “research club.”
Kathy said her center in the Global Security Division historically has had the lowest rate of LDRD idea submissions, even when it was part of the National Security Division prior to the Labs’ contract change in May 2017. In fact, only one idea went to the full proposal stage last fiscal year.
This meant that an important resource for discretionary research and development, funded in part by the LDRD “tax” levied on the center’s external funding, was largely supporting R&D in other organizations. In addition, Kathy said, “we were failing to pursue new technical directions that could lead to funded programs in the future.”
The basic premise behind the center’s Research Club was to combine the small group dynamics of peer mentoring with more traditional mentoring while coaching participants at each stage of developing an idea into a research proposal.
Managers and senior managers selected 10 early- to mid-career researchers to participate in the pilot from July 2017 to May 2018. In weekly, hour-long meetings, Kathy and the other experienced “coaches” — Tian Ma, Mark Van Benthem and Kurt Larson — led participants through a curriculum that mirrored the stages of developing a research proposal, from idea generation and literature searches, to shaping ideas within the context of the LDRD call, finding an appropriate investment area, writing the idea proposal, presenting to proposal selection teams and executing newly funded projects.
LDRD homework
There was “homework” between meetings, such as reading through the LDRD call, thinking about research topics and conducting literature searches. Each club member, individually or with a partner, was expected to submit an idea proposal for the FY19 LDRD call.
The researchers also learned more about Sandia’s LDRD process and the “Heilmeier Catechism,” a set of questions developed in the 1970s by former Defense Advanced Research Projects Agency director George Heilmeier to help agency officials evaluate proposed research programs.
Heilmeier Catechism
- What is the problem? Why is it hard?
- How is it solved today, and what are the limits of current practice?
- What is the new technical idea, and why can we succeed now?
- Who cares? If you succeed, what is the impact?
- What are the risks?
- How much will it cost?
- How long will it take?
- What are the mid-term and final “exams” to check for success?
“If you can answer those questions about your proposal and explain this to the review team, you’ll be in good shape,” said computer scientist Jenny Galasso, a member of the inaugural Research Club.
Jenny, who has been at Sandia since 2001, had not heard of the Heilmeier principles prior to participating in the Research Club, nor had she previously participated in the LDRD program. To her, the most valuable part of the club was the access it provided to the three coaches, who provided feedback and guidance at every stage in the development of her proposal.
“I got out of it a thousand times more than I expected,” she said.
Jenny said it was helpful to learn how to reach out to technical experts around the Labs to get input on her idea. The LDRD proposal process involves an internal peer review. “It’s good to advertise your proposal in advance,” she said. “Those reviewing your proposal should be aware of it in advance because they often don’t have a lot of time to review it.”
Her proposal, “Machine Learning of Signal Patterns for Protocol Informatics,” was selected to receive funding through Exploratory Express, a program for short-term feasibility assessments for high-risk, high-payoff R&D. Jenny said the club was instrumental in helping her develop her idea, which arose out of an opportunity she saw to combine machine learning and radio-frequency signal processing. Her project is completed, and the work is feeding into a follow-on LDRD project in another center in the National Security Programs Division.
‘A great sounding board’
Research engineer Matt Dykstra said the Research Club provided a great sounding board for bouncing ideas off a broader cross-section of the center than he would normally encounter in his daily work. Another benefit was the ability to have team members and coaches review his proposal and provide feedback on his presentation.
Matt’s LDRD proposal, “Metrology for Distributed Sensing Formations,” successfully navigated the LDRD funding process and will receive $1.18M in funding over three years. As principal investigator, Matt will lead a team of five on the project, which seeks to develop a real-time tracking and prediction capability for satellites. Had his manager, Amy Shrouf, not recommended that he participate in the club, Matt said he likely would not have gotten involved in the LDRD program.
“The Research Club is good for getting people who have not considered LDRD to look into it and get help developing a proposal,” he said. “It’s also a good way of bringing people together to talk about things we’re interested in.”
Expansion plans
Research Club members developed six research ideas. Of these, four were submitted in response to the FY19 LDRD call for proposals, and two of these were funded. Two additional ideas were funded by external program sponsors before the FY19 LDRD call was issued.
As a result of this success and the positive feedback from participants, the Research Club was expanded in FY19. Three Research Clubs will run in parallel: two groups from Global Security kicked off in October, and a third group with members from the National Security Programs division and the computing research center began meeting in late November. To enhance cross-organizational networking and collaboration, each club will include about a dozen members from multiple centers. The three groups will meet weekly through June, which is when oral presentations for full LDRD proposals are conducted.
When Kathy, who was appointed a Sandia Fellow in September, approached Associate Labs Director Doug Bruder about expanding the club, he immediately said yes.
“We want to maintain a culture at Sandia where employees can recognize a need and take the initiative to formulate and experiment with new ideas,” Doug said. “A challenge for leadership is to find ways to sup-port and encourage these emerging successes without overregulating them. In this case, over a third of employees have been at Sandia five years or less, so there is a growing need for initiatives such as the Research Club that bolster our research culture and support knowledge transfer.”
John Zepper, director of Kathy’s center, said the Research Club concept dovetails nicely with the new Sandia strategic priority “Unleash the power of Sandia.” He credits Kathy’s passion for Sandia’s mission and her ability to mentor and channel people’s ideas with the Research Club’s success.
“The Research Club helps leverage the newness and excitement of the newer folks at the Labs and gives them the tools to succeed. Now that we’ve had local success with this concept, let’s share it.”