Agency
Name:
National
Science Foundation
Agency
Tracking Number:
1139623
Panel Summary
Panel
Summary
Intellectual Merit:
The panel found the project overall to be well intentioned with a worthwhile
focus. Panelists highlighted several strengths, inlcuding
the fact that the project was building on successful earlier work, the prior
experience of the PI with science teams in the district, the focus on early
elementary school (3rd grade) learners, the inclusion of community college
students as mentors, and the willingness by schools to participate (as
evidenced by several letters of support).
Despite these positives, however, the panel also pointed out some significant
weaknesses:
-The project description lacked sufficient detail, especially with regard to
mentor and teacher trainings and the classroom implementation component. This
made it difficult to see how all the various pieces of the proposed project
would fit together.
-The robotics activities presented seemed very prescriptive in nature, which does
not seem in keeping with an emphasis on the development of 21st century
learning (i.e., critical thinking, creativity, communication, collaboration,
etc.).
-Within the evaluation plan, project goals and objectives were not clearly
connected to research/evaluation questions, and these in turn were not clearly
linked to data sources and measures. Also, credentials of the external
evaluator were not provided.
-There was no clear committment by the higher
education institutions that will be brought into the project after the first
year.
-The proposal lacked plans for both dissemination and sustainability.
The panel felt that carefully addressing the issues mentioned above and perhaps
drawing more descriptively on the lessons learned and outcomes of the prior
experience of the PI with robotics in the elementary school would significantly
strengthen the proposal.
Broader Impacts:
Panelists thought the focus on early elementary (3rd grade) to be crucial to
creating a pipeline as it would contribute to
establishing a strong STEM culture in the schools. The choice of community
college students as mentors was seen as promoting K-16 STEM understanding and
partnership and perhaps fostering interest in STEM teaching among community
college students.
Summary:
The core ideas and intent of the project (to build student interest and teacher
capacity in STEM learning) are laudable and the PI obviously has a vision for
enacting the program. The proposal, however, falls short of communicating that
vision effectively. The project description needs to make explicit each of the
components of the project as well as provide an overview of how those elements
fit together. Equally important will be the careful construction of an
evaluation/research plan that can provide formative input and document the
successes and challenges over the life of the project.
PANEL RECOMMENDATION: Non Competitive
Summary of all Reviews
Grant Application All Reviews: 1139623
Agency
Name:
National
Science Foundation
Agency
Tracking Number:
1139623
Organization:
NSF
Program:
Innovative
Technology Experiences for Students and Teachers (ITEST)
PI/PD:
Nicholson,
King
Application
Title:
A
Strategy for The Integration of Robotic Competitions & Activities Into a
Rural K12 System Through Partnerships with a Local Community College: A Model
for Statewide Expansion
Review 1
Rating:
Fair
Review:
What is the intellectual merit of the proposed activity?
The proposed project builds on the PI's multi-year
experience working with college students at Central Alabama Community College
who visit precollege classrooms and do physics lessons. This is an interesting
idea, which enlarges the horizons of the precollege students and college
students alike. (I wonder how many of CACC students who have participated ended
up being teachers.)
Developing capacity in supporting robotics activities in-class and
out-of-class, first in the vicinity of CACC and then ultimately throughout the
state of Alabama, is a laudable goal. The way in which the project plans to
spread out is well thought out and cost-effective.
The fact that the program starts in third grade is a strength
of the proposal.
There are some serious weaknesses in the project description. The activities,
as described, sound quite prescriptive and cook-book.
What expertise do the teacher leaders need to have to ignite creativity among
the students? How will they develop that expertise? What will the focus of
professional development and support be for these
teachers? Who will provide that? How will results of research on the learning
and teaching of science and mathematics find their way into the classroom? I do
not doubt that there are 31 pages connecting Lego activities with academic
concepts but just because there is "a line" of relevant text in both
does not mean that the materials are "aligned" with state standards
or best practices.
The research questions are not all aligned with evaluation activities. For
instance, RQ6 (arguably one of the most ponderous questions) is not addressed
at all in the evaluation. Also, this reviewer knows firsthand that journaling
is a very teacher intensive (and very worthwhile) activity. What types of
journaling prompts will there be? What scaffolds will the students get? Who
will provide the teachers with the specialized training that is needed?
What are the broader impacts of the proposed activity?
The project will impact up to 1000 students in an area that has significant
need for such an activity. Yet, the focus of the narrative seems to be more on
implementation than on making a lasting contribution to the knowledge base of
successful strategies. No specific dissemination plans are outlined.
Summary Statement
Review 2
Rating:
Excellent
Review:
What is the intellectual merit of the proposed activity?
What is the intellectual merit of the proposed activity? Student, educators and
curriculum developers will gain a greater understanding of the use of robotics
to teach STEM related topics. The use of First Lego League will challenge the
team to advance their STEM concepts learned through real world activities. The
participation in the FLL and after school contests will allow the team to
experience other possible solutions to these problems as well.
˜ How important is the proposed activity to advancing
knowledge and understanding within its own field or across different fields?
The use of secondary school lead teachers and science educator teams in each
school will help saturate the schools with the STEM concepts proposed.
Developing the STEM program with students, interdisciplinary post-secondary
professionals and teachers in class rooms together with students will help
transform understanding across the various STEM related fields as well. The use
of undergraduate students will help infuse the various fields into real world
robotic related activities and help students gain a greater understanding of
potential STEM careers.
˜ How well qualified is the proposer (individual or
team) to conduct the project? (If appropriate, the reviewer will comment on the
quality of prior work.) The professional team has many real world experiences
as well as sufficient grant experience to see this project through to completion.One memver
of the team is a NSF Fellow.
˜ To what extent does the proposed activity suggest
and explore creative, original, or potentially transformative concepts? Many
schools across the nation are doing this transformative activity currently. The
aggressive plan this proposal contains is integrating competitions, STEM
concepts and class room integration and growing regional program that will
serve as models for a possible statewide roll out.
˜ How well conceived and organized is the proposed
activity? This proposed activity was very well conceived and organized.
˜ Is there sufficient access to resources? The
activity will have more than sufficient access to resources through First Lego
League, after school activities, Science teams and online resources are more
than sufficient.
What are the broader impacts of the proposed activity?
What are the broader impacts of the proposed activity? The schools involved
will engage in many STEM activities that will develop students and educator
STEM knowledge. The development of successful regional STEM schools will help
the eventual full integration of STEM concepts in many area programs as well.
˜ How well does the activity advance discovery and
understanding while promoting teaching, training, and learning? Students will
also be exposed to hands on STEM and problem solving activities that will
stimulate greater understanding and potential understanding of STEM careers
through these activities.
˜ How well does the proposed activity broaden the
participation of underrepresented groups (such as gender, ethnicity,
disability, geographic, etc.)? This grant request will broaden many
underrepresented groups in this region. By involving female and other
underrepresented students in the lower grades these groups will gain a greater
knowledge of the potential in STEM career fields.
˜ To what extent will it enhance the infrastructure
for research and education, such as facilities, instrumentation, networks, and
partnerships? The local students, college and secondary instructors will work
together and collaborate on physical and curricular resources to create a
network that should sustain the program beyond the lifetime of the grant. The
tools (Lego Mindstorms) and computers requested need
to be increased to ensure enough equipment to help greater integration of STEM
knowledge.
˜ Will the results be disseminated broadly to enhance
scientific and technological understanding? The program if successful will
disseminate the scientific and technological understanding to the state of
Alabama.
˜ What may be the benefits of the proposed activity to
society? The benefits of the grant application will be greater participation by
the communities' underrepresented population with greater understanding of
potential STEM careers available to them. The educators and professionals
involved will also be enabling them to transform their curriculum to better
educate students in STEM education and careers. Student achievement should
display positive testing results. The use of team participation and the in
First Lego League competition will give students and educators many resources
and potential concepts to continuously improve their STEM program.
Summary Statement
The overall grant was well written and there is much academic and STEM educator
support demonstrated. The concept and curriculum is well documented and has
been proven effective in other programs nationally. It is clearly demonstrated
what the testing mechanisms would look like. The STEM content delivery looks to
effective. The grant seems to possess well balanced
equipment versus curriculum ratio with intense immersion in STEM concepts.
There is evidence of need for this training in the state of Alabama because of
low graduation rates and under represented non-traditional populations. Much
evidence was displayed that the collaborative team are experienced with various
successful grants and the team includes a NSF Fellow. The goal is lofty and far reaching to expand the grant throughout the state. The
use of lead teachers and science teams experienced in contests and project based learning is displayed throughout the grant.
The equipment costs and content should be sufficient to allow students and
schools to immerse students in STEM activities. The use of First Lego League
and 4C's, (critical thinking, communication, creativity and collaboration)
concepts will enhance the transference of STEM concepts to the class room and students for greater understanding. I believe
that through the collaboration with the Central Alabama Community College,
First Lego League competitions, after school contests and schools training will
be well balanced amongst the 4 STEM components. A positive component of the
proposal is the recognition that the grant seekers need to aggressively seek
additional revenue sources to enhance their aggressive program. This outreach
will help with community "buy in" and help sustain the program well
past the 3 year period.
The use of under graduate lab assistance, lead teachers, science teams and
college specialist will help strengthen the integration of STEM knowledge and
understanding. Students will gain valuable STEM knowledge and experiences
through readily available "hands on" robotic and computer equipment.
There should be sufficient transference of skills and knowledge experience with
the minimum of three students per computer and individual robot kits. Overall,
the grant should be an excellent mechanism to integrate STEM education and
knowledge into the applicant's community. The three year
systematic development of strong programs in each region is an aggressive
program. The process they plan to develop a strong program in each region each
additional year should reap satisfactory benefits for all groups involved.
Students will be exposed through immersion into various STEM fields. What a
tremendous opportunity for the students of Alabama.
Review 3
Rating:
Very Good
Review:
What is the intellectual merit of the proposed activity?
The goal of this project is to provide the resources necessary to establish
classroom robotic activities and after school robotic competitions for a
consortium of school districts linked to the Central Alabama Community College
District. The target group is students starting in the 3rd grade through the
12th grade. A hands-on laboratory experience at a young age is the theory being
implemented and tested. The pedagogy employed will be engagement in active
exploration, interpretation and construction of ideas. The partnership between
the Community College and all the Districts in the region has much merit,
especially in an area of much need and lack of achievement. Training for a
substantial number of high level administrators, lead
teachers and science teachers will build expertise. Adequate resources will
enable teachers to assist students in the learning of the concepts. The after
school competition will allow students to work cooperatively and demonstrate
what they have learned. The curriculum to be utilized is readily available and
will not have to be developed. The science teams who are in charge of the after
school competitions will require paid students to work with lower level students.
This will provide a more sustainable model of program continuance. The team
seems well prepared and well suited for this collaborative activity.
What are the broader impacts of the proposed activity?
An evaluation of this activity will allow for a study on the effectiveness of
this type of program to leverage state policy makers in the expansion of this
program. The goal is to have this type of program and competition set up state
wide with adequate local and state funding. This area is unfortunately known
for over 50% dropout rate and requires an innovative, systemic intervention.
The intention of beginning at the 3rd grade will provide the foundational
interest level for long term sustenance. The systemic
approach targets necessary grade spans and involves the higher educational
institutions, junior colleges and universities as the catalyst for improvement.
Curriculum is readily available which allows for quick scale up if successful.
Also, the professional development at each level will build capacity for
continued internal growth and external supports necessary to assist other
regions in their start ups. The timeline begins with
five K-12 systems and follows up with the addition of three regions in each of
the following two years. The evaluation portion incorporates formative and
summative aspects, with an experimental design approach. Longitudinal data,
multiple quantitative and qualitative methods will be used with triangulation
coming from web based evaluation instruments. Dissemination will be shared by
the online database. It will be shared and reviewed internally and externally
in written and verbal formats for a variety of audiences.
Summary Statement
I rate this proposal Very Good. The area of focus is a high need area, with the
targeted underrepresented students as the main beneficiaries. The grade level
span beginning in 3rd grade allows for exposure when student interest is very
high and builds a necessary foundation. The ability to scale this program is
very good and the goal is to make this statewide. Sustainability is based on a
solid presentation of the findings to the state and local level policymakers in
charge of funding. The systemic approach with K-12 working with the two levels
of higher education, junior colleges and universities, offers a unique
opportunity for these institutions to work together to develop this program,
rather than the traditional top down approach that lacks appropriate results.
This model offers opportunity when successful to other areas of the nation that
find themselves with similar populations and the same types of results. This
type of systemic change at the K-16 levels involved can do much to address high
dropout rates, better representation for new majority students in STEM careers
and quality systems focused on the success of underrepresented students.
Review 4
Rating:
Fair
Review:
What is the intellectual merit of the proposed activity?
This project seeks to provide resources to establish an in-class robotics
program and after school competitions. The project does target multiple school
districts and up to 1,000 students in grades 3-12, providing training for 7
regional directors (physics instructors at the regional college), 70 Lead
Teachers and 108 team members (paid college students who will coach the
robotics competitions).
The project builds upon the PI's thirteen years of
experience sponsoring student science teams working with third grade classes on
science activities over the course of the year. The current project could
benefit from the lessons learned and cost effective materials developed for
elementary classrooms in prior years as described by the PI. The several
letters of support from schools to be involved and the PI's
experience in the system may facilitate the implementation of a larger in-class
program.
The proposal includes several examples of specific in-class activities
(presumably used in the past) and lists the robotics kits to be used, but
provides no detail regarding the program design, its components, or how it will
be implemented in classrooms. Curriculum development for the in-class program
appears to be left to participating classroom teachers without a clear process
(or standards) for development and ensuring quality. There is also no
description of the training that will be provided to Directors, Lead Teachers
and college physics students who serve as mentors for the K-12 students in
their classroom and in after school competitions.
The proposal might be strengthened by a detailed description of the previous
program and the specific ways in which the current project diverges or builds
on prior work, a rationale for changes/decisions made, etc.
What are the broader impacts of the proposed activity?
The project seeks to build teacher capacity and cultivate k-12 interest in STEM
in a region that is severely disadvantaged economically.
The proposal states several laudable outcomes and objectives related to
increasing K-12 student interest and understanding of STEM content and careers
and college student mentors gaining confidence (in what areas is not stated)
and increasing leadership skills. However it is difficult to determined
the specific new knowledge that the project seeks to contribute to the field.
The evaluation plan is confusing and lacks coherence. Evaluation and research
terms are mixed throughout without presenting a clear design, theory of action,
or logic model to understand what and how the project will be learning from its
efforts. The research questions are broad and uneven in their scope and most do
not seem answerable through the data sources described in the proposal's plan.
The Evaluation Plan Timeline is typically a place to link evaluation questions
to specific evaluation activities and instruments and an administration
timeline. Unfortunately, the matrix and the explanation of instruments that
follows do not provide an understanding of the nature of the tools and what
data they are designed to collect.
Summary Statement
The PI has several years of experience with a similar program and letters of
support accompanying the proposal bode well for the partnership proposed.
However, the lack of detail describing the program key components (student
science team and teacher training; curriculum development for the in-class
program) and its implementation make it difficult to discern the potential
innovation or feasibility of the project. By the same token, the lack of a
clear research or evaluation plan with well defined learning objectives and indicators suggests that the project will not be able to
provide new important knowledge to the field.