EVALUATION CRITERIA

The IAIN has formulated the following guidelines to be used in identifying and incorporating aerospace and aerospace-related sources of information to be included in the IAIN Webpage.

The following general guidelines are provided to determine resources to include in the IAIN:

• Information resources should be included only if they contain substantive information of relevance to the aerospace community.
• Personal web pages or simply collections of pointers to other resources normally will be excluded. However, collections of resources equivalent to a resource guide (offering at least some evaluative and descriptive information in addition to a catalog of links), or single aerospace-related organizations’ pages offering one stop resource lists on specific topics, or home pages of institutions or their departments, often constitute valuable resources in the own right and should be included.
• Material that is strictly local in context, out of date, or no longer available should not be included.

SPECIFIC GUIDELINES

These specific guidelines are intended for guidance purposes only. In the final analysis, it will be one's overall impression of the value of a resource to the aerospace community which will decide whether it should be selected for inclusion in the IAIN.

1. ESTABLISHING CONTEXT

A. SCOPE

• Is the subject area, breadth, depth, time period and format or type of information covered relevant to the aerospace community?

B. AUDIENCE

• Who is the intended audience of the information in the Uniform Resource Locator (URL), and does this affect the suitability of the resource for the aerospace community?

C. AUTHORITY

• Has an individual or institution taken clear and unambiguous responsibility for the resource? Material unsigned by an author cannot easily be assessed as to the authority of the information content.

• Is it possible to ascertain the status, qualifications or reputation of the author?

• Is the publisher a reputable, recognized organization?

• Does the resource document the sources the information is based on and how that information was obtained?

• Is the resource sponsored in any way or funded by grants?

• Is an e-mail discussion list or newsgroup moderated?

• Is an electronic journal refereed?

• Is an electronic journal indexed? (where?)

• Has the resource been reviewed elsewhere? (is that source reputable?)

• Does a recognized professional association or a specialist information service link to the resource?

For many resources, the authority of the publishing organization will carry sufficient weight to allow one to select a resource. Information from peer reviewed journals, government agencies, national and international aerospace organizations and research centers is generally valued highly.

If a resource is provided or sponsored by a recognized institution, or the author is providing the information in their capacity as employee of such an institution, there may be no need to establish authority further.

D. PROVENANCE

• How long has a resource been available (either in print or as an electronic version)?

• If a print equivalent exists, how well established is that?

• Are there any indications of some established history and continued maintenance?

• How extensive are the archives of electronic journals or discussion lists or newsgroups? Are they available for retrospective searching.

Indicators such as these can help you select resources which are sustainable.

2. ASSESSING CONTENT

A. COVERAGE

• Does the resource cover a subject adequately? Are there inexplicable omissions?

• Is coverage integral, or is the resource part of a greater whole?

• Does the resource contain substantive information or is it simple a list of links? If links are provided are these evaluated or annotated in any way?

B. ACCURACY OF INFORMATION CONTENT

• Is the information factual, or opinion?

• Can its objectivity be assessed? Is there any evidence that it may be representing vested interests or undeclared biases?

• Is it possible to determine the accuracy of the information provided?

Only subject experts can often answer questions of information accuracy; nonetheless, positive evaluations of authority and provenance can provide strong indications of likely accuracy.

C. UNIQUENESS/COMPARISON W1TH OTHER SOURCES

• Is the resource original, or has it been derived from other sources? Are these documented?

• Does it complement another resource, for instance by providing updates to a print source?

• Is this an original or a mirror site? Is the resource mirrored elsewhere? How do they compare?

• Are there any print or other equivalents (e.g. CD-ROM) (e.g. for electronic journals, are illustrations available as well as text)? How do they compare with the print original?

• If the resource is available in different formats, how do they compare? (e.g. FTP-able resource may be available as ASCII, pdf or Postscript files, etc.)

• If mirrored, does a resource provide the same extent of coverage (retrospectively, e.g. archives, as well as current), currency of data, ease of use?

• Does it provide special features, unavailable from the original site? Is the user interface significantly different?

Much of the evaluation process can be applied comparatively to identical or near identical alternative versions of a resource. It. may be possible to identify the original resource to establish which mirror sites are recommended.

D. CURRENCY/FREQUENCY AND REGULARITY OF UPDATING

• Is the resource static or reliant on regular updating.

• Can the currency of the information be ascertained? Is it possible to assess the currency relative to another source?

• In the case of a document, is a date given?

• For software, is there a version number?

• How often is a resource updated? Is a policy for maintenance stated (e.g. frequency of updates)?

• How current is the material included in each update?

• If a resource is meant to be updated regularly, how reliable is the updating? (e.g. for a weekly electronic journal, are updates as regular as expected? If it is mirrored, might it fall behind schedule? If there is a choice, how does one source compare with another?)

• Are time sensitive resources (e g, news) available in near real time?

3. ACCESS ISSUES

A. ACCESSIBILITY

• Is the resource easily accessible, at least some times of the day?

• Is access to a resource reliable or is it intermittent due to server maintenance?

• Is there a charge to access the resource? How do charges compare with alternative sources?

If there are access restrictions:

• Is a resource available only during certain specific access times?

• Do geographical access restrictions apply?

• Is special registration or institutional subscription required?

If there are special requirements:

• Has the information provider allowed for different modes of access to a resource?

• Is use of a resource possible at all with a text mode browser, e.g. Adobe Acrobat.

• Are there special software or hardware requirements? (e.g. software available only by Macs)

B. USE OF GRAPHICS

Are images used appropriately (e.g. are thumbnail images used)? or are they merely decorative?

For an electronic journal: are tables and graphical material included? If not, are they referenced and are captions included?

C. DESIGN AND LAYOUT/USER INTERFACE

• Is the general layout of a resource functional. Are menus, headings and formatting used effectively?

• Are navigation aids available to guide users?

• Are hypertext links used appropriately? Are they relevant? Are they maintained?

• Is a simple search facility available?

• Is a full complement of search options available?

• Is the search speed acceptable?

• Is a choice of display formats available?

• For a database, is there a consistent record layout? Is the data presented in a standard fashion (e.g. not in all-upper-case)? Are the data elements easily distinguishable? Are they tagged consistently? Can the data be recommended for importing into spreadsheets or reference management software?

Although the appearance and functionality of the interface of a resource will have a significant bearing on the overall impression a resource creates, it is primarily the value of a resource in terms of information content that you are likely to be most concerned with; unless, of course, usability is significantly compromised by design or interface factors.

D. USER SUPPORT/DOCUMENTATION

• Is the resource accompanied by an introductory or explanatory material (e.g. home page or parent document) or FAQs or README files? Does this outline policies regarding the scope and coverage of a resource, currency and maintenance?

• Is on screen help available? Is it context sensitive?

• Is there a contact name and e-mail address for further advice and information?

• Is on-line or print documentation available? Is it accurate and clear? Is it available free or at a nominal cost?

Most resources accessible in the network environment are likely to offer a modicum of user friendliness. Some, though, may be sufficiently complex that an element of user support would be needed to make the best use of them.

Please note: as a rule the information source should make point to useful resources wherever they appear in the hierarchy of a site, i.e., they are linked at the level of individual web page of interest, rather than merely the top level Homepage. The exceptions to this would be where resources do not stand on their own, i.e. individual discussion items from a Newsgroup or Listserv or article from an electronic journal, unless they are of particular significance or importance.

IAIN will normally point to the homepages of electronic journals and archives or FAQs of email discussion lists and newsgroups