Developing Tools

Shifts from single-species management and traditional conservation planning to EBM are rapidly increasing the scope and complexity of EBM tools. For most of the past 20 years, tool development has focused on relatively simple stock assessment models and conservation site selection algorithms. EBM tools, however, need to account for a wide range of factors, particularly dynamic ecosystem processes and socioeconomic factors.
To develop flexible tools that will be most useful to scientists and managers, tool developers should think through the entire tool development and deployment process and develop tools that will be interoperable with a range of other tools.

Best Practices for Developing and Deploying EBM Software Tools

Tool Conceptualization and Functional Specifications

Improved awareness of which tools already exist, how they are structured, and how they are being used can help both developers and users be better prepared when embarking on a new tool development project or determining whether a new tool is needed.

 Recommendations and Cautions

  • Make tools that fill gaps.  Scan all available databases and existing reviews of tools.  Don’t repeat what others have done.  It is simply a waste of time and money.  Search the EBM Tools Network Tools Database. 
  • Keep the specifications simple, focused, and transparent.  After the tool is created, you will have to repeatedly explain what it does and how it works.  Seek a balance between offering a useful tool and a tool that tries to do too much and does none of them well.  Develop interoperability with other tools that perform functions well rather than duplicating that functionality in your tool.
  • Create tools for a specific audience with a specific need and involve those intended users in the design.  Creating tools without a thorough understanding of the need can lead to products that are underused or abandoned.

Software Architecture, Programming, Interface Design, Versioning

New tool development, from the software developer’s perspective, is full of a range of choices as well as pitfalls.  Increased awareness of existing tools, as well as being in contact with a network of developers, can help overcome common problems.

 Recommendations and Cautions

  • Consult with a network of practitioners working within your general area and outside of your own work environment before beginning.   Avoid working in isolation from other tool developers and users.  Search the EBM Tools Network Tools Database to learn about other tools and developers in your general area.
  • Use off-the-shelf functionality where possible.  Even simple functionality can probably be bought at a cheaper cost that it can be created and debugged.  Don’t create “home grown” software where there is a solution already available “out of the box”.  The tendency is toward gross underestimates of both time and resources required to create custom solutions.
  • Focus on enhancing interoperability with other tools.  Avoid using proprietary input and output formats whenever possible.  Learn about best practices for developing interoperable tools.

Documentation

Tool documentation is required and can be oriented towards three different audiences:  1) End users, 2) Technical professionals/programmers, and 3) Scientific staff.  It can take the form of comprehensive self- help documentation, tutorials, or quick tips and tricks.  It should cover all of the analytical assumptions and limitations inherent within the tool as well as all known issues with using the tool.  

Recommendations and Cautions

  • Be very explicit regarding tool assumptions, limitations, and uncertainties inherent within the tool and the inputs.  Do not oversell the applicability of the tool.  You do not want to create or reinforce the false idea that the tool replaces human decision makers.
  • Make critical information easy to find.  People will likely read this as a last resort and they want the information to be fast and easy to access.  Users do not like to read documentation, especially lengthy sections where key information is not readily found and digested.
  • Use visual and graphical information where possible.  Consult with a visual design professional for help in this arena.  Don’t create a text-heavy or formula-heavy document if at all possible.   

Establishing credibility

Tool users want to know that the results from using the tool are accurate, relevant for decision making, and transparent.  This credibility is established through documentation, testing and validation, peer reviewed literature, and published case studies of application.  Validation and case studies made independent of the tool developers will enhance a tool’s credibility.

 Recommendations and Cautions

  • Validate the tool results by encouraging or conducting research to test the results and assumptions in the real world.  Don’t ignore the fact the many tools have a long list of established publications that back their use and, that to be competitive, your tool will have to do the same.
  • Be prepared to backup all spatial analyses and formulas used in the tool with peer-reviewed literature.   Avoid creating your own, brand new, ways of doing an analysis if there is an established, credible, and equally-effective method that you can adopt.  It is ultimately easier and cheaper than designing new research for validation and testing.
  • Try to establish and cultivate a strong user community that can generate case studies of use of the tool.  Don’t assume that you and your development team will be able to establish the credibility of your tool alone.

Maintenance

Who will maintain and host the tool?  How much time, resources, and money are required for long term upkeep and maintenance?   How will customer feedback be collected, organized, and addressed? How will updates, bug fixes, and the release of new versions be handled?  When should a tool project call it quits?   All of these questions must be answered for any group that wishes to support and maintain a tool over the long term.

Recommendations and Cautions

  • Develop a plan for long-term maintenance and upgrades of the tool.
  • Have a good way to track bugs, feedback, and suggested improvements to the tool.  Consider using a system that is specifically designed for this type of work.  Don’t assume that you can track all of the user comments through email or spreadsheets.  This will quickly become unmanageable.  
  • Budget significant programmer and project manager time for maintenance.  Don’t assume that maintenance will be cheap or be done on an ad-hoc basis.  Other priorities will always take precedent without clear dedicated resources.

Marketing and Communications

A tool investment is not worth making if potential users never find out about the tool.  From Web presence to publications and conferences, marketing and communication builds the awareness that will eventually increase the appropriate application of tools in critical coastal management applications.

 Recommendations and Cautions

  • Develop a strong Web presence for your tool.  This ideally would be a place where users can learn about your tool and share information with other users.  Good Web design requires a significant investment of time and energy.
  • It is worth the expenditure to enlist both visual design and content design experts in the creation of the Web presence for your tool.  Web sites that are not well designed and don’t include up to date information can damage the reputation of your tool.
  • Establish some early, relevant, and successful case studies that demonstrate the tool’s utility.  Potential users are unlikely to spend the time learning and applying a tool without some track record.
  • Develop outreach materials tailored to the practices and language of your audience(s).  A message mismatched with the audience will be ineffective.

Training

Training is essential to establishing and growing a user base for a new tool.  A number of options exist for training new users on tools:  1) Tutorials and e-learning, 2) Standard training classes, 3) Customized training classes, and 4) One-on-one instruction and technical support.

 Recommendations and Cautions

  • Evaluate what training is needed to use your tool. Use early adopters and custom one-on-one training to understand training needs and appropriate training offerings.  Training must be designed for a specific audience.  A training that is poorly designed for the audience can sometime diminish interest in a tool.  Misjudging training needs may lead to a number of failed applications of the tool.
  • Dedicate significant resources to development of a training program for your tool.  It will pay off in greater use and acceptance of your tool.  Don’t assume that you can develop training as an afterthought.  Developing a great training program will require almost as much time as developing a great tool.
  • Develop a formal training program for your tool and use the knowledge of professional learning designers and trainers if at all possible.  Knowledge of adult learning styles and techniques is critical to effective training.  Don’t assume that the tool development team will create the tool training by themselves, since they will rarely have the full range of skills to design and deliver effective training. 

Field Deployment and Support

What does it take to get a tool from its first release to it being used regularly in a real world setting and having a positive impact on the decision making process it was designed to address?  It requires a keen understanding of how what kinds of human and technical resources are required to deploy a tool in the field.  Understanding and planning this process is a crucial step for any organization that wishes to promote the appropriate use and application of tools.

 Recommendations and Cautions

  • When embarking on the field deployment of a tool, make sure that you have an interdisciplinary team of people who understand not only the technical aspects of running the tool, but also includes science staff who can assist in asking the appropriate questions.  Be sure to clearly understand the decision making process you are influencing.  Many efforts fail to properly integrate tools into the decision making process because the interaction with the tool was limited to GIS professionals or other technical staff.  It takes an interdisciplinary team to truly integrate tools into decision making.   
  • Focus on stakeholder engagement.  You must have a strategy to communicate the pertinent results of the tool to your stakeholder group.  You must accomplish this without unduly burdening them with the technical aspects of the tool.  It is important to include people on your field deployment team with specific knowledge and experience with stakeholder engagement.  Stakeholder groups will quickly reject a tool if it is perceived to be irrelevant to decision needs or overly complicated.  It is important that stakeholders feel that the tool is there to serve their needs and not intrusive to the process.

Best Practices for Developing Interoperable EBM Software Tools

Ecosystem-based management involves integrating information and methodologies from diverse fields (such as ecology and economics), sectors (conservation, industry), and sources (scientific studies, traditional knowledge). It can also involve integrating diverse software tools, such as data acquisition tools, ecosystem models, conservation site prioritization tools, and stakeholder engagement tools. By designing tools to be easily linked—i.e., by making them “interoperable”—developers can ease the complexity of implementing EBM projects. The following best practices describe steps that tool developers can take to improve the interoperability of their tools.

Address interoperability early in the development cycle
Developing a tool that interoperates with others is more complicated than developing one that operates by itself. Maximize your chance for success by designing for interoperability from the start, rather than trying to add it on later. Allocate a larger budget than you would for a stand-alone tool, both for initial development and for ongoing maintenance. Recruit developers who have experience developing interoperable software.

Select an appropriate interoperability method
Consider a simple EBM project that links two tools in a workflow: A → B There are several popular methods for linking the tools: 

  • Direct invocation – in this method, tool A programmatically invokes and controls B to accomplish a specific task. B may be a stand-alone tool capable of running on its own or it may be a programming library, component, or web service expressly designed to be invoked by another tool. In any case, the developer of B usually designs the linkage and the developer of A must adhere to it.
  • Plug-ins – in this method, tool A is designed to allow other tools to be "plugged in" to carry out specific tasks in its internal workflow. B is a "plug-in". As above, A programmatically invokes and controls B, but with this method, the developer of A designs the linkage and the developer of B must adhere to it.
  • Workflow components – in this method, each tool is designed to plug into a workflow system such as the ArcGIS Geoprocessing Modeler or the Kepler scientific workflow system. The workflow system invokes and controls each tool and is responsible for passing data between them. The workflow system determines many aspects of the linkage between the tools and the developers focus mostly on the data passed between them. These data structures are often defined by the developer of whichever tool is written first, and adhered to by the developer of the second tool.
  • Manual data exchange – in this method, the user is responsible for manually executing the tools and passing data between them. Tool A usually outputs files or other data structures that can be read by B. As with workflow components, the data structures are often defined by the developer of whichever tool is written first, and adhered to by the developer of the second tool.

Many considerations can factor into your choice of an interoperability method. Sometimes you will have no choice because you're trying to interoperate with an existing tool. When you do have a choice, consider these points:

  • Manual data exchange is the most popular method and is often the easiest to implement, but it imposes more work on the tool user, who must coordinate the execution of the tools. Linking tools via this technique can also be laborious for programmers when the data structures are in obscure formats.
  • The direct invocation and the plug-in methods can allow seamless integration of the tools and the highest degree of customization of the linkage, but often require highly skilled developers to develop properly. The linkage can break if developers do not carefully coordinate changes to their tools. 
  • Workflow components are often a good compromise between direct invocation and manual data exchange, particularly for tools that do not require user interaction. If your users regularly use a workflow system (such as the ArcGIS Geoprocessing Modeler), consider implementing your tool as a workflow component.

When in doubt, consult an experienced software developer. If possible, consult the developers of the tools that yours will interoperate with.

Define and fully document a formal interoperability interface 
After selecting an interoperability method, identify the specific interoperability scenarios that you will support and those that you won't. Think through all of the details of interactions between your tool and those it links with. Write a formal specification of those details, including the sequential steps in the supported scenarios, the names and parameters of any functions your tool exposes to others, the complete description of data formats written and read by your tool, and so on. These details constitute the interoperability interface for your tool, and the document is called an interface specification. Use widely-adopted, stable technologies and data formats in your interface Select from the technologies and data formats that are most commonly encountered today. Do not select obsolete technologies or formats. Resist the temptation to select brand new ones if doing so would impose an unnecessary learning or programming burden on the developers of tools that will interoperate with yours. If possible, contact those developers and ask what they would prefer.  

Provide interoperability examples
Most developers prefer to start from an example rather than an interface specification. Start by providing the simplest example possible, like the classic "Hello, world" program. Then provide more complex examples that illustrate the most common interoperability scenarios.

Make your source code available to other developers
Ideally, your interface specification should be complete enough that it answers all questions that a developer may have. But occasionally there are questions that can only be resolved by looking at your source code (or talking to you). If you do release your source code, you should select a licensing scheme compatible with the goals of your organization. See www.opensource.org/licenses for more information on open source licenses.

Manage changes to your interface very carefully
When you change your interface, it is very easy to break tools that interoperate with it. For example, if you change the parameters of a function, any code that calls that function will fail until it also is changed. Rather than changing the existing function, consider adding a new function that includes the new parameters. Keep both functions in your interface for a while, until your callers have migrated to the new one. Then remove the old one. When you must make a "breaking change," document it very clearly and notify developers that you know will be affected.

Choose your tool's dependencies very carefully
When your tool takes a dependency on an operating system, programming library, or other software, it forces this dependency upon other tools that interoperate with it. Carefully consider whether your tool's dependencies are satisfactory to the tools that interoperate with yours, and to the users of those tools. Select widely-adopted, stable technologies. Try to minimize the monetary cost and installation and operation complexity imposed on users.

Be careful with UI
If your tool has a user interface (UI), it is likely that developers of tools that interoperate with yours will want to replace or alter your UI. If your tool performs calculations, conducts a simulation, or something like that, separate the UI code from the computational code. Expose the computational functionality from your interoperability interface, so that other tools can present their own UI and then invoke your tool to do the computations. If your tool must present a UI, seek feedback from the developers of the tools that invoke yours.

Write high quality code
When another developer takes a dependency on your tool, he does so because he thinks he can save time. His dream is that your tool will solve some problems for him so he can forget about those problems completely. He doesn't know your code and he doesn't want to. When your code is defective, he usually can't fix it and must work around it. This can be extremely time-consuming and frustrating.

Test your interoperability interface rigorously
Write fake tools that interact with your tool through its interface and exercise its functionality as completely as possible. Run these programs whenever you produce a new version of your tool. If you know of real tools that interoperate with yours, run these tools and verify that they still work. Enlist the aid of the developers of those tools to run tests before you release a new version of your own tool.

Plan and test your packaging and installation strategy
If your tool is invoked by another tool, decide up front whether you will allow or require that tool to package yours with its setup program, or if the user must perform two separate installations. If you will support your tool being packaged by others, be sure to document how this should be done. Determine whether you will allow a multiple versions of your tool to be installed on the machine at the same time. If you will not allow multiple versions, plan how you will handle the "DLL Hell" problem, in which a program invokes your tool but it fails because the user has installed a newer version of your tool than that program was designed to invoke. Give developers of other tools a stable method for identifying the version of your tool that is installed on the machine. If your tool invokes other tools, determine whether it is permissible and feasible to package that tool with your setup program, and consider the other issues mentioned above.

Provide clear, actionable error messages
Do not assume that underlying libraries that you call will report meaningful error messages. Trap errors and report clear, actionable messages.

Provide your email archive, bug list, and source code online When you have a problem calling a function written by someone else and can't find a solution in the documentation for that function, you can often find the answer by searching the Internet. The solution often appears in a message posted to an email list, a forum posting, a bug description, or even the source code of the function you're calling. Make these sources of information available for your own tool by posting them on a web site.

Support developers who contact you 
Clearly specify how to get in contact with you. When a developer contacts you, respond as quickly as possible. Plan to support developers after your project has completed. This may require a budget for ongoing maintenance. If you are unable to provide support anymore, clearly say so.

[These best practices were developed by Jason J. Roberts from Duke University with input from EBM Tools Network members.]