Directions in Electronic Portfolio Development

David Gibson
National Institute for Community Innovations, USA
Vermont Institutes, USA

Helen Barrett
University of Alaska Anchorage, USA
International Society for Technology in Education, USA


ABSTRACT. This article explores the advantages and trade-offs of two paths to the development of electronic portfolios – using generic tools and using customized systems. The analysis assumes that the goal of electronic portfolios is to stay focused on the quality of work by a learner and the valid alignment of their work to the standards and goals of education. A comparison of the “best of all worlds” scenarios for both approaches shows that the two approaches each have important strengths that need to be weighed during the process of program planning.

Introduction. Portfolios have been around a long time. Educators have used them long before the digital age, which suggests that perhaps we can look at the new media and medium of electronic portfolios in part, through lenses from the past. For example, in the past, portfolios were assembled from collections of work stored in boxes or three-ring binders, and now, are stored in digital form. Electronic portfolios, according to Barrett (2002), are essentially a new kind of container and can be developed along two paths. One path uses generic tools (GT) such as word processing, HTML editors, multimedia authoring tools, portable document format (PDF), and other commonly used productivity tool software. The second path uses an "information technology” customized systems approaches (CS) that involve servers, programming, and databases.

What are the advantages and trade-offs of these two paths if the goal of electronic portfolios is to stay focused on the quality of work by a learner and its valid alignment to the standards and goals of education? By what criteria can we compare the two approaches?

The significance of these questions arises in more than the technological sense of which methods best preserve the heart of traditional creative, reflective portfolio work. Because the new medium is so flexible, it can easily be used for practices that can work counter to the traditional goals of portfolios. For example, once they are digital and more easily stored and searched, portfolios might be used as high stakes gatekeepers, like standardized tests of today. Just because digital portfolios can be used this way, should they? Our discussion of directions in e-portfolio development should help us see the differences in the political, human and technological dimensions of our decisions so we can make better informed decisions, recognizing the trade-offs, strengths and weaknesses.

Criteria for Comparisons. To begin the conversation about pluses and minuses of each kind of path, Barrett (2002) offers the following. The ability to aggregate data for assessment is counted as a plus for CS and a minus for GT, and the low start-up and maintenance costs counts as a plus of GT (leaving us to question whether it is a plus or minus for CS). She also points to Mary Diez’ (1996) conception of the portfolio as “mirror, map, and sonnet.” These metaphors give us hints about possible criteria for the comparison of GT and CS approaches. The mirror concerns the portfolio’s reflective nature that allows us to see our own growth over time. The map includes the portfolio’s ability to aid us in planning, setting goals, and navigating the artifacts we create and collect. And the sonnet points to the portfolio’s role as a framework for creative expression, encouraging diversity within a template or structure for thinking about work and presenting it to others.

Also suggestive of criteria for the comparison of GT and CS, Davies (2002) notes the benefits of e-portfolios as providing flexibility in three important respects: 1. Organizational flexibility, 2. Display flexibility of content and ideas and 3. Ability to connect content to various schemas for representation in multiple ways, such as to standards, key concepts, interdisciplinary connections as well as linking works to one another. To these we could add Communications and Collaboration Tools, since with e-portfolios, we have not only a new container for work, but also a new process for producing, improving, and sharing it. Beyond these issues are considerations of the ways that e-portfolios actually change thinking and action through both the media and the processes of supported reflection in learning. This gives us a starting point for discussing the pluses and minuses of GT and CS as directions in electronic portfolio development, a list of criteria for comparing approaches.

  • Planning & Goal Setting
  • Moving Toward a Rubric. To facilitate a comparison on each of the above criteria, we’ll imagine a continuum with pure CS at one end and pure GT at the other, recognizing that this is an abstraction for the convenience of debate.  A simplistic definition of the two approaches which we’ll use to begin: A pure CS, one might imagine, does not require the learner to know anything about HTML, nor to actively organize or link work. It can be thought of as essentially a single on-or-offline application that to some extent allows and depends upon external applications for the production of work products. We use the term “Application” in rubrics presented below to describe the CS approach, and distinguish that from an educational program that uses the GT approach. The imagined CS application maximizes cross-portfolio comparisons and in general seems more “top-down,” controlled by an educational program. Much of the process in a CS system - the flow of work and creation of linkages and documentation - takes place within the application, with most of the work products being made externally to the application using generic tools. Assessment management systems tend to fall on this end of the spectrum, with the goal of creating a digital record of achievement, especially a record that program requirements have been met. In an educational environment controlled by accountability concerns, CS systems help administrators prove that their programs are producing the desired results.

    A pure GT approach on the other hand, we can also imagine, requires the learner to learn and use multimedia tools and HTML or an equivalent language or process for producing the digital products and a final display or collection. The learner in a pure GT approach starts with a blank slate and must construct an entirely original representation and expression of their work, resulting in unique collections that are difficult to compare from learner to learner. The GT approach can be thought of as an assortment of external applications that are called upon to construct the electronic collection and documentation of work. In general, this approach seems more “bottom-up” controlled by a learner. Much of the process in a GT approach is in private work and face-to-face communications (with asynchronous communications as a possibility) within an educational program. The process takes place without any overarching application and utilizes many separate applications to create or document work for digital collections and to form the collection itself. We use the term “Program” in the rubrics that follow to distinguish the GT from the CS approach. Examples of products of the GT approach abound in faculty web pages on the Internet, especially in higher education systems where the institution’s primary support amounts to access for transferring files to a server.

    Between these idealized extremes, many electronic portfolio projects in education are hybrids, borrowing from both approaches as they create their own unique solutions. In what follows, are some thoughts in a rubric framework where the essential criteria are described as fully developed, in both the CS and GT approaches, offering a way to see the e-portfolio development process as a series of choices and capabilities. A full discussion of the criteria, with developmental stages outlined for minimal and partially developed systems is forthcoming (Gibson & Barrett, in press).

    The set of criteria might be applied to an existing e-portfolio product or program to help raise questions about the purposes, supports, and desired next steps in establishing development directions.

    What we present here is a broad framework displaying all of the criteria and the best possible conditions given each type of approach (the GT generic tools or CS customized system approach). Following that broad overview, we present each criteria separately, with some suggested scales from “minimally present” to “mixed” to “fully developed.”

    We are seeking input from the field on our general approach, the system of criteria we are proposing, and the specific guiding language used in the draft rubrics.

    Planning & Goal Setting

    One benefit of a portfolio can be its use as a tool for planning self-directed learning or setting learning goals, either by the learner, or an advisor, or both.
      Minimally Present Mixed Fully Developed
    Customized System Application has no specific structure for facilitating the planning and goal setting process The application’s tools can generically support either planning or goal setting, by either (but not all) of the following: prompts, documentation, and linkages to other parts of the system. Planning processes are prompted, synchronous or asynchronous conversations are documented, goals can be flexibly linked to standards and other frames of reference determined either by the organization or the individual.
    Generic Tools

    (e.g. scheduling, timelines, project management systems, visualization, databases, spreadsheets)

    Planning takes place “off line” and artifacts of the process are not expected in the portfolio. Documentation of planning and the evolution of goal setting are acceptable content for portfolios. Expectations include the digital documentation and portfolio presence of planning and goal setting and adjustments as part of the story of growth over time.

    Framework for Creativity

    Based on Mary Diez’ metaphor of the portfolio as “sonnet”, does the approach allow a learner to express their own creativity within the structure of the portfolio?
     
     
      Minimally Present Mixed Fully Developed
    Customized System Learners may have few if any tools for creatively building or editing their portfolios, or have few if any choices to make. Learners are able to exercise limited choices when building portfolios.  The application allows learners to customize all digital products. Learners either have CS tools or are expected to use GT to add creatively to their portfolios.
    Generic Tools

    (e.g. tools for visualization, animation, audio, video, databases, spreadsheet representations, graphic production)

    Inflexible templates or stock multimedia elements (sounds, graphics, logos) are used by learners for the organization and display of their portfolios. Learners are encouraged to create some original elements or organizational aspects of their portfolios. Learners are taught and supported in the development of rich and varied, expressive multimedia skills. Portfolios display the individual creativity of each learner.

    Communications

    Does the approach use telecommunications to facilitate communication between learners and advisors/faculty?
      Minimally Present Mixed Fully Developed
    Customized System Application does not utilize telecommunications in its processes or documentation. Application allows some use of telecommunications or allows documentation of such in portfolios. Application integrates asynchronous and synchronous communications into all processes and documentation is available to be used in portfolios.
    Generic Tools

    (e.g. email, threaded discussions, video conference systems, webcasts)

    Program does not include telecommunications in its processes or documentation. Some telecommunications are used to develop plans, goals, work products, and the creation of portfolios. Some learners document their communications for inclusion in portfolios. Portfolios show evidence of use of telecommunication tools in planning, goal setting, work improvement over time, and final products. 

    Collaboration Tools

    Does the approach provide tools that support collaboration between learners and advisors or faculty?
     
      Minimally Present Mixed Fully Developed
    Customized System Application provides few if any collaboration tools. Collaboration tools are limited to basic communication, not co-creation of work and documentation. Collaboration may be supported for either the process of assembly or the improvement feedback on the work products or portfolios. Application supports multiple group and individual roles and relationships that support self, peer and expert co-creation and dialog about portfolios and their products.
    Generic Tools

    (e.g. threaded discussions, net meetings, video conferences, whiteboards, asynchronous work spaces)

    Program does not emphasize or there is little evidence of collaboration in portfolios. Program uses some generic tools for collaborative work and encourages learners to include evidence of collaboration in at least one portfolio.  Documentation from generic collaboration tools is prompted and supported in all portfolios. 
    Reflective Processes

    The portfolio should provide an opportunity for learner reflections on artifacts as well as how they support multiple purposes (goals or standards). Does the approach support reflection to address multiple purposes and audiences?
      Minimally Present Mixed Fully Developed
    Customized System Application has few if any prompts or other supports for creating reflections on work. Reflections can be created and attached to the application, with or without built-in tools.  Application prompts for and supports multimedia reflections on work and the creation of alignment between purposes and audiences for multiple portfolios.
    Generic Tools

    (e.g. word processor, video, audio, multimedia production)

    Written or audio reflections primarily deal with the alignment of work to program requirements or personal statements. Reflections using multimedia expression are encouraged. Alignment of purpose and audience may have a single focus and reside in one portfolio. (e.g. a “graduation” portfolio demonstrating that standards have been addressed.) Learners are collaboratively assisted to reflect and create alignment of purpose and audience in more than one portfolio, ideally, a working folio, a program completion folio, and one or more other folios for employment, public and private purposes.
    Connection Capabilities

    In what follows a “schema” is meant as a framework that represents standards or other organizational constructs that are useful to organize the presentation of portfolio products. An electronic portfolio should take advantage of the linking capabilities of digital technology, to make easy connections between various work products, the learner’s reflections or rationale statements, and the various schemas. Does the approach provide the capacity for the learner to make linkages according to the various purposes or audiences?
      Minimally Present Mixed Fully Developed
    Customized System Application does not provide ways to link work products to schemas. Built in linkages may be available, or learners may add some limited linkages as needed. Application facilitates maximum use of linkages among and between work products and other representations and multiple sets of schemas. Learners have flexible access to the linkages to make adjustments and create new connections.
    Generic Tools

    (e.g. hypertext capabilities in word processors or publication tools, web page applications, raw HTML)

    Some learners invent their own ways of making a few linkages to a schema. Several learners make some linkages to or publish their work alongside at least one schema. Learners are expected to extensively link their work to more than one schema, depending upon audience and purpose of a portfolio.

    Organizational Flexibility

    The flexibility of the portfolio’s organization will determine its usefulness to meet multiple purposes. Is the portfolio approach flexible enough to facilitate the multiple organization frameworks?
     
      Minimally Present Mixed Fully Developed
    Customized System An organizational framework may not be present, or may be present but inflexible or limited in it customization possibilities. One or more organizational frameworks are possible, but are limited in flexibility. Multiple frameworks are supported and can be deployed flexibly across learner work areas and portfolios.
    Generic Tools

    (e.g. hypertext capabilities, databases)

    Learners use few if any hypertext or database capabilities to flexibly reorganize their work. Methods of flexible organization are taught and encouraged, but not expected of all learners. All learners maintain more than one way to organize their work collections and utilize more than one organizational framework to represent their work.
    Display Flexibility and Transportability

    The flexibility in being able to display the portfolio in multiple formats will determine its usefulness to meet multiple purposes. Is the portfolio approach flexible enough to display the work in multiple formats? Is the data only accessible in a specific system, or can the portfolio be transported to other formats?
    Minimally Present Mixed Fully Developed
    Customized System Few if any choices for display are offered to the learner within the application. Final product displays are limited in scope, flexibility or transferability. Data is only accessible from system and cannot be transferred into a stand-alone format. Application supports some dynamic displays and offers choices among templates for presentations. Final products are either extensible in scope, flexibility or transferability. Application supports multiple representations as well as the use of either internal or external tools for enhancing the display of work. Final products can be saved to hard drive storage, CD, DVD and other digital as well as hybrid print/media formats.
    Generic Tools

    (e.g. many tools have display possibilities, advanced uses include database driven web displays, active server pages, and dynamic HTML)

    Display of works is essentially the same from page to page or slide to slide. Generic tools are used with their most basic default capabilities. Generic tools are used with some of their more advanced hypermedia  features. Portfolios show evidence that students can use the advanced hypermedia features of generic tools to create flexible or dynamic displays of their work. Final format is portable and transferable in digital format.
    Data and Information

    Portfolios used as part of a program evaluation process provide the opportunity to aggregate data across program objectives and learners. Does the approach address learner privacy and intellectual property issues? Does the approach facilitate the development of a comprehensive picture of each individual student as well as the capacity to aggregate data across an institution or program? Do the organization’s needs for assessment data interfere with the learner’s need to present a rich and individual representation of their growth over time?
      Minimally Present Mixed Fully Developed
    Customized System Few if any data collection methods are provided. Application may raise concerns about storage and the permanence and ownership/privacy of information. Data collection, analysis or reporting methods are partially supported, with few concerns about storage, permanence and ownership/privacy of information. Client may not be able to impact the data and information system. Application supports multiple data collection, analysis and reporting processes across learners and allows records to be kept and displayed over time. Storage, permanence and ownership/privacy issues are solved flexibly with the client.
    Generic Tools

    (e.g. databases, spreadsheets, visualization tools, GIS, web searches, virtual libraries)

    Portfolios give a limited picture of the student in terms of their own intentions for learning and the programs’ intentions for their learning. Portfolios give a reasonably valid and detailed picture of some aspects of the student’s learning and show some of the balance of program and individual intentions of learning. Each portfolio is a rich, valid and balanced picture of an individual student (their intentions in learning balanced with the program’s intentions for their learning) that is in part, not commensurable with other students.
    Start-up Costs and Maintenance

    The portfolio approach needs to be cost-effective to be useful and used. What are the start-up and maintenance costs of the approach, from the viewpoint of both the organization and the individual learner? Does the system provide support to learners and faculty/advisors, as well as IT staff?
     
      Minimally Present Mixed Fully Developed
    Customized System Implementation of the application places high demands on an organization for servers, software, maintenance, and programming staff. Implementation of the application gives an organization choices for the burden on servers, software, maintenance and programming staff, but the costs are based on business models and expenses and generally favor large scale users. Implementation of the application offers several flexible options for servers, software, maintenance and programming staff that are low cost and sustainable even for small sites.
    Generic Tools

    (servers, system software, lab licenses)

    Program has little or low centralized support for applications, updates, server space and access, multiple licenses for products from uncoordinated buying across the organization. Program provides periodic support with a few options for training, but the type and depth of support places a high burden on some people, creating barriers to ubiquitous implementation. Program has a continuously improving IT support that is client centered on the learning program and all of its members. IT provides low cost group purchases with automatic updates of the software suite, provides on-demand and continuous training and support to both learners and teachers.

    Which is better, the Generic Tools or the Customized System approach?

    Hopefully, the brief picture presented here helps show that both approaches have strengths, depending on the degree to which a particular criteria has been addressed by both application developers and the educational program. Either approach can stand alone, but they may be weaker for doing so. A CS approach by itself soon loses touch with the individuality of inquiry and expression of learners. A GT approach by itself limits its contribution to a program’s validity as well as accountability.

    Which approach is better also depends upon the purpose and audience for the information within and connected to a learner’s portfolios. Note that the word “portfolios” is plural, pointing to the fact that for a body of work by a learner, there can be several portfolios or sharing collections, each aimed at a difference audience for different purposes. For example, in a private working portfolio, the audience is the learner and the purpose is to improve work over time. Some of the strengths and advantages of the CS approach are irrelevant to this portfolio, for example, comparisons across portfolios of other learners. Not only does the comparison violate the privacy of a private working portfolio, but also the information that could be obtained from other portfolios is unlikely to be comparable. Imagine, for example, trying to compare a budding composer’s private working folio with that of a future lawyer; not only the content and intentions would be radically different, the symbolic notations and language would be incomparable. Even within a single discipline area when two people have agreed to try to represent the same subject using a common style and starting point, great differences in each person’s private intentions are practically guaranteed to lead to divergent results. The divergence of thought and expression is a good thing for the expansion of knowledge through the free play of the creative mind!

    An example from the other perspective is evident when we ask about the impacts of a program on its graduates. For example, are candidates being well prepared to teach and lead in the schools and communities where they will find work? As we frame answers to this kind of question and ask to see products of student work that demonstrate that we are indeed meeting such a challenge, common frameworks for development and assessment of work seem to be an obvious criteria. In a public portfolio demonstrating the impact of the program, some of the strengths of the GT approach are not helpful; for example, if individual creative expression makes it difficult to determine the impact, or creates an unfair advantage or disadvantage for making that determination. Imagine assessing a lesson plan and a dance for evidence of an ability to plan instruction to meet the needs of all students. Not impossible perhaps, but such divergence soon stretches public credibility.

    Conclusion. The realistic alternative is then an interpenetrating combination of both approaches with appropriate caution about which approach is driving the process in relation to purpose and audience. A program and its developers cannot confuse systems accountability issues with portfolio reviews and feedback to learners. Nor can they afford to confuse individual expression with achievement, or privacy with isolation of information. These issues can be reconciled within a complex system where roles and access are appropriately determined for different audiences for different times and purposes.

    The benefits of taking on the challenge of integrating both CS and GT into a comprehensive e-portfolio development program are numerous. Rich pictures of learning in incommensurably individual expressions are as useful as highly comparable measures of performance and achievement and both are needed for a deep understanding of learners and a program’s impact on them. Individual performances linked to commonly experienced program opportunities help tell the organization’s story of impact better than either the numbers or the pictures by themselves. Use of both centrally held and widely distributed capabilities can strengthen all users of data to create information. Appropriate use of generic as well as customized system tools builds technological, critical and creative thinking skills.

    The emphasis on which tools and how deep they are developed varies with the context as well as the challenge facing the learner and their supporters. The success of that emphasis depends as much on how a program and its developer partners are communicating and co-developing as on the static qualities of the generic tools or customized system approaches.

    References

    Barrett, Helen (2002) Presentation, ISTE’s Forum on Assessment and Technology, San Antonio, TX, June 14, 2002.

    Davies (2002) Presentation, ISTE’s Forum on Assessment and Technology, San Antonio, TX, June 14, 2002.

    Diez, Mary (1994) “The Portfolio: Sonnet, Mirror and Map” in Burke, Kay, ed. (1996) Professional Portfolios. Skylight Training & Publishing

    Friedrichs, A. & Gibson, D. (2002) Personalization and Secondary School Renewal. In "Personal Learning: Preparing High School Students to Create Their Future." Scarecrow Press.

    Jonassen, David (1999) Computers as Mind Tools for Schools (2nd Edition) Prentice-Hall



    ITFORUM PAPER #66 - Directions in Electronic Portfolio Development by David Gibson and Helen Barrett. Posted on ITFORUM on November 30, 2002. The author retains all copyrights of this work. Used on ITFORUM by permission of the author. Visit the ITFORUM WWW Home Page at http://itforum.coe.uga.edu/