Contract for: Accompanying Measures

1 Project summary *

2 Project objective(s) *

3 Participant list *

4 Contribution to programme/key action objectives *

5 Membership *

6 Community added value and contribution to EU policies *

7 Contribution to Community social objectives *

8 Economic development and Science & Technology prospects *

9 Workplan *

10 Clustering *

1. Project summary

The project aims at operating a Network of Excellence for HLT, which builds on the existing ESPRIT-LTR ELSNET. The network will support the HLT industrial and research communities by means of a number of actions and services, focusing on transfer of knowledge and expertise (e.g. training, information dissemination, referral services), evaluation (towards a European evaluation infrastructure), resources (exploration of new language resource types and annotation schemes), best practice and standards (repositories), and R&D road maps (technology watch, research challenges and priorities). There will be a special focus on interdisciplinary actions, and international collaboration with non-EU countries, especially new accession countries. The fact that ELSNET already exists as a fully operational network, with 136 key academic and industrial players as its members, will ensure broad support from the field, and a seamless transition to FP5.

 

2. Project objective(s)

The main objective of the project is to advance human language technologies in a broad sense by means of the operation of a network (ELSNET) encompassing a broad cross-section of private and public research centres actively engaged in language and speech technology research, development, integration or deployment.

The network's role is to offer an environment that allows for optimal exploitation of the available human and intellectual resources in order to advance the development and deployment of language and speech technology. This environment comprises a number of structures, actions and services, which all aim at supporting the implementation of the IST programme in general, and the Human Language Technologies (HLT) area in Key Action III in particular.

There are many ways to decompose the HLT universe, but from the ELSNET point of view, the following six dimensions are crucial for the further advancement of the HLT field:

1. Transfer of knowledge and expertise. This comprises a variety of flows of knowledge and expertise, e.g. between academia and industry (in both directions), between languages (porting technologies from one language to another), across disciplines, or from developers to "deployers". This dimension forms the basis of our training, dissemination and referral services.
2. Shared goals. In order to create contexts for joint efforts to face major challenges, ELSNET will produce road maps for various sub-fields of HLT, which will help establishing common goals and priorities.
3. Evaluation. In order for researchers and developers to be able to monitor their own progress, also in comparison with others using different approaches and methods, it is important that there are facilities to support this. ELSNET will offer a platform for discussion and further dissemination of evaluation expertise and experience on both sides of the Atlantic.
4. Shared language resources. Language resources are essential, both to support research and development, and to support evaluation. As new types of application areas emerge (e.g. multimodal applications), new types of resources will be needed. ELSNET will actively explore the desirability and feasibility of possible new types of resources required by new applications, in close concertation with other bodies involved in the production and distribution of language resources.
5. Promotion of best practice. For any potential technology integrator or deployer, it is important to have immediate access to what counts as best practice. Progress (both from the technologist's and from the user's point of view) is hampered when every developer, integrator or deployer has to rediscover which technologies can be safely used for his purposes. ELSNET aims at becoming a repository for best practices in the field.
6. Interoperability by means of standardisation. One important condition for being able to join forces within a community is the existence and usage of proper standards, ensuring at least interoperability between systems, data and components. ELSNET will actively engage in the promotion of emerging and de facto standards.

The above six dimensions span ELSNET's area of activity. Throughout the next two years we will be active along all six dimensions, but some of them will be more prominent in our work programme than others, both for strategic reasons (some activities are expected to have more strategic impact than others), and for practical reasons, such as the fact that the EC is already funding projects in these areas.

Top priority will be given to action lines (i) and (ii), where we intend to play a proactive role. As for action lines (iii) – (vi), we will concentrate on collecting and disseminating information and results, fostering international collaboration and information exchange (both with other high-tech countries and with outlying regions), and acting as a nursery for innovative initiatives.

In the following sections we will describe the nature of ELSNET's activities and services in more detail. Furthermore we will give an account of the organisational structures established offered to support this work.

1. Participant list

The ELSNET consortium consists of the three main partners – the University of Utrecht (Co-ordinator), VDI/VDE-IT and ARAX (principal contractors) – and a potentially very large number of "member" organisations.

At the time of writing, an initial group of members has already been identified – and made known to the EC – whose proved excellence in the field and longstanding association with the network forms the basis of ELSNET's operational core. Additional members will be introduced into the network throughout the duration of the project, and to this end a pro-active campaign will be conducted in the first 3-6 months of the project, which is expected to yield over 140 adhesions.

Membership agreements will be drawn up in accordance with FP5 standard texts. An updated list of members will be presented to the EC project officer every six months.

Note that the Accompanying Measures framework was retained, as opposed to say Thematic Networks, given the fact that the latter does not allow to implement the required scientific, technical and prospective work, and the task delegation process which has always characterised a large and evolving group such as ELSNET. Besides, ELSNET membership agreements must clearly distinguish between normal members, who have no contractual tie with the EC, and operative members (see below), charged with specific tasks stipulated in this Annex.

Operative Members

The Executive Board (cf. 9.7), in consultation with the EC services, will entrust a number of member organisations – henceforth referred to as operative members – with the performance of specific tasks through subcontract agreements, in order to maximise the impact of the network on the target business and research communities. The criteria for the choice of operative members will be proven excellence and authority in the relevant research and industrial circles.

The EC project officer will be fully informed before drawing up contract agreements for operative members, and his opinion will be sought prior to the signature of any subcontracts. Addenda to the main contract with the EC will be considered whenever appropriate (e.g. in case of sizeable subcontracts relating to new groups or events) and in agreement with the EC services. One such contract addendum is scheduled for the beginning of the second year of the project.

Subcontracts for operative members (which are meant to be organisations, not individuals) will detail intended tasks, deliverables and time scales, and the associated acceptance and payment modalities. They are meant to cover basic work and travel requirements: the work carried out by operative members is expected to exceed in value the cost stipulated in the subcontract. The coordinator and the Executive Board will establish standard terms of reference and codes of conduct for subcontracts, and investigate the use of 'harmonised' daily rates in line with those used by the EC for its task-contract letters.

Operative members expected to take the lead in tasks scheduled to start within the first 2-3 months of the project are: CNRS-IDF Sud-LIMSI (Centre National de la Recherche Scientifique, Orsay, F), UPI (University of Pisa, I), SDU (University of Southern Denmark, DK), IMS (Universität Stuttgart, D) and UoB (University of Bergen, N). The following organisations are expected to join in as operative members in the first 3 to 6 months of the project: KTH (Stockholm, S), University of Sussex (GB), University of Patras (GR), UPM (Madrid, ES). Others have already been invited to participate for the organisation of specific events: ILSP (Athens, GR), Charles University (Prague, CZ). Yet others will be involved as work progresses. Further details are provided in chapter 9.6.

 

2. Contribution to programme/key action objectives

Networks of Excellence have first been introduced in 1990 as part of ESPRIT's Basic Research Actions, and have been continued in ESPRIT Long Term Research in the Fourth Framework Programme. An evaluation of the concept of Network of Excellence by an independent panel of experts, conducted in 1996, led to the following conclusions:

• The experience of existing Networks of Excellence (NoEs) shows that the concept is sound and is a cost-effective way to create frameworks for co-ordinating R&D and related activities between key industrial and academic players in a technological domain. Networks of Excellence have demonstrated that it is possible to mobilise a community of researchers and engineers around long term technological goals and to operate, in a co-ordinated mode, towards their achievement.

• NoEs have been successful in promoting knowledge and information exchange and therefore learning by experience sharing

• NoEs have contributed to the structuring and progress, at various levels, of R&D and related activities in IT in Europe.

ELSNET (set up as the European Network of Excellence in Language and Speech), which became operational in 1992 as one of the three pilot networks, has evolved from a small consortium of academic participants to a major network with 136 participating research institutions and private companies, all active in the field of language and speech technology.

It unites Europe's key players in the field, and as such it constitutes an enormous pool of expertise, which can serve both its own members and other parties, provided there is a proper infrastructure to share and channel this expertise.

Over the years, ELSNET has gradually built up such an operational infrastructure, which (although funded by ESPRIT) has supported participants in both the ESPRIT and the Language Engineering programmes, and will thus be ready to support the HLT programme and constituencies from day one.

The desirability of such a network and its scope are clearly stated in the IST documents and need no further justification here. From the six 'pillars' mentioned above and the more detailed presentation in the following sections it may be clear that the present project covers the elements listed in the IST and HLT documentation (and even a few more, which we feel are indispensable for optimum exploitation of the benefits of a network of this type).

Yet it should be noted that the present project does not aim at just copying the existing ELSNET to the new framework programme. Significant changes are caused by the new focus of HLT R&D (e.g. human-orientedness, multilinguality, multimodality), by the increasing need to internationalise, and by ELSNET's wish to professionalize more of its services.

1. Membership

ELSNET has currently 136 member sites, and is gradually extending itself; 85 members are academic or public research institutions, 51 are industrial. The network covers at this moment 26 European countries, cf. the table below:

EU Countries	Member sites		Other Countries	Member sites
Austria	3		Belarus	1
Belgium	4		Bulgaria	1
Denmark	4		Czech Republic	1
Finland	2		Georgia	1
France	15		Hungary	3
Germany	21		Lithuania	1
Greece	3		Norway	2
Ireland	2		Poland	2
Italy	10		Romania	1
Portugal	2		Russia	3
Spain	6		Switzerland	2
Sweden	3		Ukraine	1
The Netherlands	15
United Kingdom	27
Total sites	117		Total sites	19

Academic institutes in language and speech can be admitted to the network if they are internationally recognised and have sufficient critical mass:

• The group should be internationally recognised, with at least 3 long-term staff engaged in a relevant research area; it should normally have a PhD programme or be prepared to accept placements of students who are carrying out PhD research; and it should be taking part in international research projects;
• it should submit a written account of recent research activities and a list of selected publications in the area;
• it should submit an account of its future research strategy and how this fits in with the goals of the Network;
• it should undertake to use (or take steps to obtain) electronic communication facilities (i.e. at least email and access to www) for Network activities;
• it should declare itself willing to actively participate in the activities of the Network, and indicate the nature of its possible contribution (e.g. membership of committees, participation in workshops, organising or hosting events).

Industrial members can be admitted if they are actively engaged in language and speech, and commit themselves to contributing to the activities of the network. The nature of the contribution may vary, and could include e.g. offering student traineeships, membership of committees, active participation in workshops, etc.

The network does not charge any membership fees.

A complete list of members is attached. If one looks at the membership list, one can observe that it covers nearly all key players in the European field of HLT.

Although at this moment all members are European (EU, associated EFTA countries, Central and Eastern Europe), it has already been decided (in close consultation with the EC) that ELSNET will also progressively open to non-European participants able and willing to contribute to ELSNET's goals, within the administrative and financial constraints imposed by EU regulations.

When looking at the new challenges posed by HLT R&D, it is clear that it would be wise not to limit membership and co-operation exclusively to those who are active in language and speech, but also to extend them to relevant neighbouring disciplines. First of all there is an increasing awareness that language and speech are not the only information carriers, and that the Information Society will make more and more use of combinations of modalities (written language, speech, graphics, vision, etc), and of transformations between modalities (e.g. for the disabled). Secondly, language and speech (and other modalities) are to an increasing degree embedded and integrated in the whole of Information and Communication Technologies, and as a consequence problems involving language and speech may have to be addressed in the context of their role in other information systems.

It follows logically that ELSNET members will not only be recruited from the language and speech communities, but also from neighbouring disciplines. This will no doubt have an innovative impact on ELSNET's activities.

1. Community added value and contribution to EU policies

There are a number of reasons why the creation and operation of a truly international network as ELSNET has an added value at the European level.

First of all: the immensity of the problem. Even if we can observe an increasing stream of new products and prototypes in language and speech technology, we have to admit that in many respects we have just begun to scratch the surface of the real problems, and only a massive research and development effort, beyond the financial capabilities of any single state in Europe can help us to make some modest steps forward. We need a considerable critical mass of technological expertise, which can only be created at the European level. As a matter of fact we want to go even further, and have included explicitly alliances with parties from outside Europe in order to better tackle the problems we are faced with.

Secondly, many of the problems we are faced with are inherently European (as opposed to national), and require solutions at the European level. With a few exceptions, the multilingual European information society is not a national creation, but a direct consequence of the fact that in a united Europe we are confronted with a linguistic diversity, for which no single state can be held responsible. The fact that a European solution of this problem will, as a spin-off, also move us far ahead in this respect at the global level, is an extra incentive for a European rather than a multiple national approach.

Thirdly, the fact that some countries are more advanced in language and speech technologies than others (for historical or other reasons) can be exploited if there are proper European infrastructures to 'port' expertise and solutions from one language to another. Many of the individual member states, particularly those where language and speech technology is relatively well developed, have long-established and successful networks within which knowledge of activities, resources, and best practice is disseminated at national level; and the dominant position of the USA in this field means that American developments are usually well-known. But there are very few channels for disseminating such knowledge between European nations. The fact that our subject matter is languages makes linguistic boundaries between European countries particularly significant barriers to exchange of ideas, resources, etc. in this domain. A body promoting communication at the European level seems indispensable for the health of the field.

 

2. Contribution to Community social objectives

First of all it is the wish of the European Union to provide every European citizen with uninhibited access to the global information society and take away all exclusion factors. It is quite clear that this can only be achieved if language barriers are removed, if communication between humans and between humans and machines is facilitated, and if information streams become manageable. The HLT programme and its support actions, such as ELSNET, aim at achieving this.

Secondly, the creation of adequate human resources via training (both for students and for professionals) will improve employment prospects and increase the competitive power of European industry.

Thirdly, proper use of human language technologies will help industries to globalise their markets, and will create new business opportunities for European industrial and service sectors.

 

3. Economic development and Science & Technology prospects

We repeat here the main dimensions of ELSNET’s area of activity, and indicate for each of them the expected impact (beyond the direct benefits for the members of the network throughout the lifetime of the project).

1. Transfer of knowledge and expertise. This comprises a variety of flows of knowledge and expertise, e.g. between academia and industry (in both directions), between languages (porting technologies from one language to another), across disciplines, or from developers to deployers. This dimension forms the basis of our training, dissemination and referral services.

Training actions will result in better-equipped human resources, which will not only have a positive effect on employability, but will also make European industry more competitive. Especially ELSNET’s focus on cross-disciplinary training will help remedy the fact that many professionals already in the field have a mono-disciplinary background, and that higher education institutions are normally not in a position to respond quickly to newly emerging needs.

Transfer of knowledge mechanisms, if set up as a continuously open channel between researchers and developers, will shorten the time from research to development, and will thus increase competitiveness.

Dissemination of results relevant for the porting of technologies and language resources to other languages will push the development of language and speech technologies for the commercially less attractive languages, and will not only improve the competitive position of these language communities, but also prevent exclusion from the information society.

Dissemination of results and achievements in general (via web or referral services) will help in creating synergies that will speed up technological development, and will also increase potential deployers’ awareness (e.g. via showcases) of the potential of the technologies, and thus help in creating new business sectors, or make existing business sectors more competitive.

2. Shared goals. In order to create contexts for joint efforts to face major challenges, ELSNET will produce road maps for various sub-fields of HLT, which will help in establishing common goals and priorities.

Shared longer term goals will help the European R&D community to concert their actions, and to make the best possible use of the intellectual resources available. They will also help in setting priorities such that the most pressing needs (as perceived by the European citizen or by European industry) will be addressed first.

3. Evaluation. In order for researchers and developers to be able to monitor their own progress, also in comparison with others using different methods, it is important that there are facilities to support this. Unfortunately there are no such facilities at the European level. As the problem of evaluation is (at least at the conceptual level) language independent, it is important that whatever information and expertise is available (on either side of the Atlantic), is shared. ELSNET will offer a platform where this information can be shared and disseminated.

This information will help researchers, developers, integrators and deployers to use systematic evaluation in order to monitor their own progress and comparatively evaluate the merits of alternative solutions. This will help identifying the most promising approaches, and thus speed up development time.

4. Shared language resources. Language resources are essential, both to support research and development, and to support evaluation. As new types of application areas emerge (e.g. multimodal applications), new types of resources will be needed. ELSNET will actively explore the desirability and feasibility of possible new types of resources required by new applications, in close concertation with other bodies involved in the production and distribution of language resources.

Language resources take a long time to develop and are expensive to produce. Annotation standards will facilitate sharing of resources and reduce cost and production time, and will hence speed up development time for applications. As the resources for tomorrow’s applications will have to be anticipated today, a proactive approach to resources will put European industry in a better competitive position.

5. Consolidation of best practice. For any potential technology integrator or deployer, it is important to have immediate access to what counts as best practice. Progress (both from the technologist's and from the user's point of view) is hampered when every developer, integrator or deployer has to rediscover which technologies can be safely used for his purposes. ELSNET aims at becoming a repository for best practices in the field.

A best practice repository will allow the potential user or deployer to have in one step immediate access to the collective expertise built up in the field. It will allow new parties to enter the arena, without having to start from scratch, and to concentrate on new products and services.

6. Interoperability by means of standardisation. One important condition for being able to join forces within a community is the existence and usage of proper standards, ensuring at least interoperability between systems, data and components. ELSNET will actively engage in the promotion of emerging and de facto standards.

To summarise, all six ELSNET action lines, even if considered separately, are expected to have a clear impact on the competitive position of European industry and business, not just for the short term, and not just for the benefit of the ELSNET members. Some of the actions may in addition have a direct beneficial effect on employability and access to the information society. Careful concertation of the actions may make the overall impact even more than the sum.

• Networks vs. RTD projects

The work plan of a Network of Excellence is inherently different from one for an RTD project, in a number of respects. First of all it should be noted that, even if from a contractual point of view a network is as temporary as any other EU funded project, from a conceptual point of view a network of this type is set up for an indeterminate period of time, and should last as long its existence provides valuable services to the community which would not have been possible without the network.

This means that the results the network is aiming at, will not be a package of completed projects, with final deliverables, but rather a collection of running services, which should continue to run after the contract period, provided there will be sufficient financial support to sustain them, either via continued EC support for the network, or by making them self-sustaining, or by separating them out and letting other parties take financial responsibility for them.

It might very well have been conceivable to set up a network with a much narrower scope, i.e. just for the lifetime of the HLT programme, and just with the task to support the projects that happen to be funded from this programme, but it has been ELSNET’s deliberate choice to aim at something more stable and lasting (as it has successfully done over the last 7 years), and to try to unite the top players in the field rather than just the ones that happen to be involved in HLT funded projects for 24 or 36 months. ELSNET’s main intellectual objective is to serve the European language and speech communities at large, and from that perspective it will be in an ideal position to use its collective expertise and excellence to provide support and services to the HLT programme in particular.

Secondly, a network is and has to be dynamic, in that it should continuously ask itself (i.e. the community it is serving) whether the services it is offering still coincide with its members' priorities, or whether new services or facilities should be taken up in order to better respond to the needs of the field, whereas other might be refocused or even suppressed.

Over the years, ELSNET has been very successful in very quickly responding to new developments in the field, and it is of crucial importance that this flexibility is maintained.

This same flexibility is also reflected by the fact that in principle all members of the network can be involved in the execution of our tasks, depending on their specific expertise and their availability. Typically, tasks are carried out by task groups, recruited from the membership, whose composition may vary over time. Unlike in standard R&D projects, the network's human resources are not just five or six institutions constituting a fixed consortium for the duration of the contract, but rather a dynamically populated cross section of the HLT community as represented by the network.

Thirdly, ELSNET has never acted as a funding agency, but as the network should not be 'monopolised' by those who happen to be in charge of its management (Executive Board, Co-ordinator, panels and working groups), at each moment in time it should be open to new initiatives from its members, for actions or services that fall within ELSNET’s scope, and that would serve the community at large.

This means that a very delicate balance has to be found between being focused and staying focused on the one hand, firmly guided by the programme of work as laid out in the network programme, and being responsive and flexible on the other.

At this moment, ELSNET is planning to set up a number of services for the HLT community, part of which are continuations of services already offered, and part of which are quite novel.

• Interaction with other groupings and communities

In Europe we find a number of parties who, like ELSNET, have as their main objective to support and facilitate R&D in language and speech technology. It is important that we maintain or establish close connections in order to co-ordinate our activities and create optimal synergies:

• IST project clusters: It is expected that the insights and results generated by these clusters will be of importance to the community at large. ELSNET will, through collaboration with the planned CLASS project (retained for funding under the 1999 IST call), provide mechanisms to give wide distribution to these results. Collaboration with CLASS will, amongst other things, include:

• ELSNET offers its dissemination channels (ELSNews, ELSNET’s web site including showcasing) for world-wide CLASS dissemination including the initiation of a community-wide discussion of the clustering principle itself;
• ELSNET will assist in recruiting relevant parties into the CLASS activities;
• The CLASS clusters will be taken as inspiration for ELSNET SIG creation;
• ELSNET will use the experience gained in CLASS for possible application within the community at large;
• CLASS will provide input to ELSNET on specific training needs;
• ELSNET and CLASS will collaborate on road map creation.
• ELSNET and CLASS will collaborate on joint events organisation.

• Other EU support projects (such as HLT AMs, Socrates TNs, FET NoEs): ELSNET will operate in close harmony with these projects, and set up mechanisms for regular consultation. Special attention will be paid to opportunities to set up joint activities with non-EU parties, both in the high-tech countries and in the outlying regions.

• Other organisations, such as professional or thematic organisations (EACL, ISCA) and ELRA: These organisations are the main organisers of scientific conferences in our field in Europe, which provide ELSNET with excellent opportunities to promote its goals. Existing collaboration with EACL and ISCA in the field of training will be continued and strengthened. The collaboration agreement with ELRA in the field of resources, evaluation and standards will be continued.

• Special Interest Groups (SIGs): ELSNET will collaborate with existing SIGs in relevant areas, and set up new ones whenever this helps us to attain our goals.

• Overall approach

The ELSNET approach to advance HLT (meant as both the field and its constituent communities) is based on the following three main ingredients:

• Set out directions and priorities for the future, so that our joint resources can be more efficiently deployed in order to bring the field forward

• Provide maximum access to our collective knowledge and expertise by
• Training
• Knowledge transfer
• Collecting and disseminating information

• Create optimum conditions for collaboration by
• Common language resources
• Common standards
• Common evaluation methods

• Work packages

The approach outlined above will be implemented by means of the following work packages:

General:

Directions and priorities:

Sharing knowledge and expertise:

Creating conditions for collaboration:

We will first briefly describe their scope, and then give an overview of the strategies and approaches adopted for each of the work packages.

1. Scope of the work packages

WP1, management, covers both the management of ELSNET as a EU funded project, and its management as a community of actors. Deliverables are typically periodical progress and financial reports. The activity will be ongoing throughout the duration of the project.

WP2 covers the self-assessment of our achievements. Deliverables include an initial assessment model and mission statement, a mid-term review report covering not only our achievements, but also our instruments and our institutional structure, and an end-of-project "balance", which will not only look back, but more importantly, also forward.

WP3, road maps, is at least philosophically the most important of our activities. It takes its basis in the fact that we unite almost all of Europe’s expertise in the field, which puts us in an excellent position to provide the community with authoritative road maps. These road maps will be based on an analysis of the state of the art, views on future developments on the market and in research, and will formulate longer-term technological goals and priorities, and intermediate milestones. Deliverables are road map reports for two distinct sub-areas, one after 12 months and one after 24 months.

WP4, training, covers all activities aimed at the training of young researchers, and of professionals already working in the field. The main characteristic of ELSNET’s involvement in training is its strong emphasis on multi-disciplinarity (i.e. integration of language, speech and other modalities). Typical deliverables are the annual ELSNET Summer Schools, and a classified and annotated collection of web-based training material in language and speech technology and neighbouring fields.

WP5, information management and dissemination, covers first of all our web-based operations, but also the other information channels we use (email lists, newsletters, reports). It covers the whole chain from information collection/generation through organisation and graphical presentation, to electronic distribution. This includes also all computer operations and information management via databases. Typical deliverables are specification of overall design and structure of our web site (month 2), operational website in month 6, and quarterly paper newsletters. It should be noted that much of the information disseminated here will be provided by the other WPs. At the end of the project a selection of the information provided on the website will be made available on a CD-ROM.

WP6, transfer of knowledge and best practices, aims at interconnecting parties with knowledge needs (providers, integrators, deployers) with either paper/electronic knowledge sources (best practice documents) or human knowledge sources (researchers with specific expertise, researchers with patentable results). This activity which will include:

• ‘passive’ brokerage in the form of a phone/fax/email/web information service and a searchable database of players and their expertise (operational in month 3); collaboration with HOPE is envisaged
• ‘active’ brokerage (experimental), whereby parties are approached individually to identify needs and results (operational in month 3, to be evaluated in month 9)
• an annotated classified collection of (pointers to) best practice documents (month 12); collaboration with projects such as DISC
• identification of the need for additional or updated best practice documents in specific areas (based on customer requests) (month 24)

WP7, resources, evaluation and standards, covers an area in which ELSNET has been very active and productive throughout its existence. These activities will be continued in close concertation with other ongoing and forthcoming EU projects. ELSNET will help disseminating the results of these projects (e.g. ISLE) and project clusters (via CLASS). It will more generally act as an international platform for exchange of information and experience, and for concertation of activities. The scope will not be limited to EU projects and programmes, but will also cover collaboration between EU and US, EU and the outlying regions, and between national projects, aiming at maximal cross-fertilisation in all directions.

We will take steps towards more synergy and concertation between national language resources projects. We will furthermore support and initiate small-scale innovative actions of strategic importance in the above fields, in close connection with our road map actions. Typical deliverables are periodical workshops in collaboration with ISLE and national projects

1. Strategy and approach of the work packages

We will here describe the various work packages and the underlying strategy and approach in more detail. As a consequence of the network structure, subtasks of the various work packages will we delegated to network members by means of subcontracts.

In section 9.6 we will summarise the various subcontracts for Year 1, and indicate which ones have already been allocated to members and which ones are still open or under discussion. Specification and allocation of subcontracts in Year 2 will be decided upon by the end of Year 1, after careful analysis of the results of Year 1, and will be the subject of an addendum to the contract with the EC.

This work package covers the creation, or rather continued existence, of the network, the co-ordination of ELSNET as an HLT accompanying measure, and the interaction with other HLT and IST support actions.

It should be noted that the starting position is not the same for all ELSNET members and associates. Whereas in the 'old' EU countries generally speaking NLP and Speech have taken off, and have reached a certain state of maturity, some other countries, such as the Central European countries about to join, and other countries with which co-operation exists, find themselves in a much more difficult position, both from an infrastructural and from an economical point of view. Although it should be clear that it cannot be ELSNET's role to provide these countries with financial support, the differences should be taken into account when setting up collaboration, and ELSNET should make an effort to help these regions set up their own infrastructures, and connect them to the rest of the community.

Similarly, following the overall philosophy of the IST programme that every European citizen should be able to participate in the information society, ELSNET will pay special attention to the interests of the commercially less attractive languages, and help ensuring that also for these languages proper knowledge transfer mechanisms are created, so that they will not risk being excluded from participation in the information society.

An important characteristic of ELSNET is that it already exists, with a membership of 136 companies and institutions, with its own working structures already in place since 1992, with some minor adaptations over the years. As this situation ensures that ELSNET can and will be operational from the very start, we are being very careful not to plan any "institutional" changes during the first year of operation. This will ensure stability and continuity for the start-up period.

At the same time it should also be noted that we are now operating in a different environment, and that some of the services detailed in the work plan are novel, either in content or in the way they are to be implemented. This may call for modifications in our working structures (cf. WP2 below). Already at the beginning of the first year the membership lists will be cleaned up in order to discard members who are no longer willing or able to contribute actively to ELSNET's objectives. Although long membership lists may look impressive, 'dead' members constitute a 'quality risk' for e.g. our planned referral services.

It should be noted that the network (as opposed to the EU sponsored project) will be managed by the ELSNET Executive Board, in collaboration with the Industrial Panel, as it has always been the case, but that the members of the Board and the Panel do not appear as project partners, as they will only be reimbursed travel and subsistence for participating in board respectively panel meetings, and not for their time or labour.

Assessment of the actions of a network like ELSNET has several dimensions:

Input/output assessment:

• Have the available resources been deployed according to our initial planning?
• Have the deliverables been delivered on time?
• Are the deliverables of the quality envisaged?

The main instrument we will use to perform assessment of this type is the periodical progress report, which will not just be an enumeration of our accomplishments, but also brief analysis of what has been done in view of our programme of work, and (if necessary) recommendations for adaptations in the way we perform our tasks and use our resources. The co-ordinator will prepare the progress reports, and the Executive Board will discuss them and decide whether they call for any changes in the way the network operates.

Institutional assessment:

Although it is not wise to continuously question and revise one’s mission and organisational structures, the transition from ESPRIT-LTR to HLT offers a good opportunity to reconsider the current structures, and investigate to what extent modifications in our structures might increase our efficiency and effectiveness. The Executive Board will carry out this institutional review after 12 months. Agreed changes will (after consultation with the EC) be implemented by month 14.

This is by far the most difficult type of assessment, as in an evolving field it is very difficult to know whether any changes in the field would have failed to happen if it had not been for ELSNET. We must accept this uncertainty, but in month 3 we will formulate an ELSNET Mission Statement, which we will try to operationalise as much as possible for the duration of the contract period. In months 12 and 24 we will make up a balance, and try to determine how much of our mission has been accomplished and which have been the main contributing factors. The results of the assessment will be taken into account when planning our actions for the next period. The impact assessment will be carried out by Board and Industrial Panel together, who will ensure that both the academic and the industrial perspective will be taken into account.

In order to arrive at a consistent and coherent approach to assessment, we will during the first

three months adopt an initial assessment model, which will be further worked out in close collaboration with our main stakeholders.

An important instrument for a network to follow progress of the technology is a so-called "technological road map", which can be seen as a global vision of the network on the future development of the technology and its take-up.

Although the focus of ELSNET’s road map activities will be on Language and Speech, and Human Language Technologies in general, we will place our actions in the context of other EU actions aiming at developing longer term visions, such as the other existing Networks of Excellence, and at a more global level ISTAG, the IST Advisory Group, whose vision is to "start creating an ambient intelligence landscape for seamless delivery of services and applications in Europe relying also upon testbeds and open source software, develop user-friendliness, and develop and converge the networking infrastructure in Europe to world-class".

The road map will include:

• an overview of market trends and needs
• an overview of the current state of the technology and current trends
• an analysis of research needed in order to make market needs and technology development converge
• intermediate milestones that help to monitor progress and success

ELSNET will not be in a financial position to commission market studies, but on the basis of publicly accessible materials, it will compile overviews for specific key areas. The choice of these areas will be made in close consultation with ELSNET's Industrial Panel and other relevant groupings.

For an overview of the current state of HLT research and technology, surveys such as the State of the Art Survey (available on the web and in book form) and its possible successor will be of crucial importance. ELSNET will actively promote and publicise such surveys via its dissemination channels.

Workshops with prominent and visionary representatives from industry and research will lead to the identification of priority research areas and intermediate milestones, which will be presented to the HLT community in the form of synthesis reports, preferably at events where they can be discussed with the relevant subparts of the communities in order to maximise consensus and support. Possible events are e.g. COLING and LREC in 2000, Eurospeech and ACL/EACL in 2001.

ELSNET has a long-standing tradition in training, with the annual European Summer School as the most prominent example. When looking at the continuous flow of announcements of workshops, schools for every season of the year, and training courses, one may wonder whether and why ELSNET should remain active in this area. The answer is clear: the ELSNET Summer School is the only training event that consistently addresses topics on the borderline of language and speech, as opposed to other schools covering either language or speech. The same applies to other training actions, where the integration of language and speech (and increasingly the integration with other modalities) are always the main topic.

A continuation of our series of Summer Schools is foreseen, possibly with the addition of other schools for specific audiences at other times of the year. Typically, these schools attract a mixed audience from academia and industry.

As these schools are typically 2-week events, and as commercial companies cannot always spare their staff for such long periods, ELSNET will also continue to offer short intensive courses on specialised topics for industrial audiences, if there is sufficient demand. ELSNET’s role will be to facilitate and promote these courses, but no direct financial involvement is foreseen, since they are expected to be self-sustaining.

ELSNET member ILSP (Athens) will organise the 2000 Summer School. The topic is Information access from text and speech. Charles University (Prague) will organise the 2001 Summer School, Annotation of text and speech corpora. The 2002 summer school itself will fall beyond the scope of this contract, but all preparations will be made. Organiser will be SDU (Odense) and the topic: Evaluation of text and speech processing systems.

If the theme(s) addressed by a given training activity is deemed to be relevant for a wider audience, course materials may be published in the form of a book or electronic publication. Currently, Kluwer is publishing the Summer School books in their series Text, Speech and Language Technology, and Kluwer has repeatedly expressed its interest in publishing books based on Summer Schools.

Part of the tasks of this WP is to create a rich web site with wide coverage of training activities and opportunities. Web based and interactive tutorials are rapidly becoming an important tool to disseminate best practise in teaching. Although it is beyond ELSNET's financial capabilities to fund courseware development directly, ELSNET will play a role in maintaining a web inventory of sites and CD-ROM productions. Such a site has already been initiated for speech communication but should be broadened to language technology in general, and its continuity can only be ensured if an organisation like ELSNET takes care of it. ELSNET will stimulate the development of new materials, and help taking steps towards a system of evaluation and certification of such materials.

For the purpose of the website the materials will be annotated and classified, but no in-depth evaluation is foreseen.

Effective information gathering, handling and dissemination is vital to the success of the project. The main objective of this work package is to maximise information sharing within the network, although most of the information will also be accessible for others, both with a view to making European efforts in the area of language and speech more visible in the outside world, and to increase general awareness of the potential of language and speech technologies and applications.

A large-scale web site shared with HOPE will be launched. The two consortia have agreed to build a single coherent infrastructure housing their respective outputs. Provisions have been made for implementing common specifications for Web and IT operations through a concerted division of tasks, within the first two months of the respective project plans. The agreed working arrangements will be forwarded to the EC Project Officer(s) for acceptance before the main implementation phase starts. The main repository of information will be the so-called HLTCentral website, which will contain up-to-date information covering several aspects of the HLT field. Further details on Web and IT related activities can be found in the HOPE work plan.

So far ELSNET has been operating a modest web service, maintained by our academic members. Given the enormous impact of the web, it is now time to enhance and professionalise this service. Although the main providers of information will remain the network itself (through members, committees, co-ordination staff), web publishing and related IT and database work will be performed out by our partners, VDI/VDE-IT and ARAX. The ELSNET web editor will be the main responsible for the organisation of the content of the website.

A technical support team (TST) at VDI/VDE-IT, supported by a specialist team at ARAX, will perform the following tasks.

At the heart of the information system will be a contacts database, to be used for project management, knowledge transfer and to support the other work packages. Email list facilities will be provided for the network as a whole and for subgroups, for internal communication and dissemination. This includes archiving and corresponding search facilities.

The goal is to provide one central contacts and profiles database maintained and used via the Web by each member of ELSNET and other related HLT projects.

The initial data will be based on existing contacts information, e.g. data from FP4 networks and projects. The final qualified CDB will consist of institutions from industry, academia and government as well as organisations such as chambers of commerce or technology transfer nodes. The database fields will cover contact data (e.g. address, telephone, e-mail, fax), short profiles based on product/service offerings, classification of institution type, a specification of size in terms of number of employees and sales volume where appropriate, a description and indexing on the base of keywords from a HLT specific multilingual thesaurus/glossary.

TST will offer the opportunity to all members to add directly to the web data base results from surveys or other sources of daily contacts with the community. TST will supervise this decentralised distributed database concept via Internet, and check on completeness of data, duplications or obsolete entries. Members who do not intend to use the data entry forms on the web will have the choice to send their raw input to TST who will put it in the CDB.

A decentralised software solution, supervised by the technical support team, will provide the consortium with up-to-date, consistent data, and an efficient and cost-effective means to run information, dissemination and awareness services. Data entries will come from the ELSNET consortium and TST will be responsible for ensuring overall quality of data.

In order to collect additional information and corroborate the validity of existing data, TST will develop questionnaires and reply forms to be distributed by e-mail. Relevant data privacy/protection rules will be applied throughout.

There will be a central ELSNET website, identifiable as such, but integrated within the HLTCentral site set up within the HOPE project, so as to achieve substantial economies of scale. The ELSNET website will be the main instrument for electronic information dissemination and exchange. The overall structural and graphical design will be part of the web support tasks. Sophisticated search facilities will be provided.

An important initial task will be the migration and consolidation of existing information (web materials, databases, etc.).

There are two quite separate roles for the ELSNET website:

• Acting as a Clearing House, providing public access to a comprehensive information centre covering many aspects of HLT.
• Acting as the entry point to the ELSNET Intranet - a password protected platform for project internal information and communication.

Planned sections (see above site map):

• HLT Results and Opportunities

Current and future R&D directions and priorities. Directories of ongoing and completed research activities, including project summaries, presentations, selected reports, multimedia showcases. The task of liasing with research centres and project teams, writing, updating, and editing presentations and reports will be carried out by ARAX.

• HLT Events and News

This section will publish the results of the scans of technical and business data (market snippets, electronic press clippings) carried out by the consortium. Information on events and trends (technologies, applications and industry; R&D policies and initiatives in the EU and in member states; worldwide developments). ELSNET contributors can fill in event information or add a web link directly or via the content editors at ARAX, who are also able to provide quality assurance.

• HLT Repository

For specific areas, such as standards and best practices, the ELSNET website will function as a repository: both passive instances (external pointers, downloadable documents) and active ones, where existing materials will have to be transformed into something more dynamic or attractive. Facilities will be provided for efficient electronic publication of reports, articles and books, such as road map reports and best practice documents. Strategic papers, studies, glossary, bibliography regarding HLT and related topics evolving within the project period. ELSNET will decide whether to put these materials on the public or private area of the web site.

• HLT Partnering

The partner search facility will focus on R&D projects and on the exploitation of their results. Spin-off opportunities evolving from successful projects within the 5^th and precursor frameworks will be highlighted as well as other existing R&D results which may need partners for the commercialisation phase.

• HLT Links

Continuously reviewed and updated links to selected HLT-related sites. The website will also feature showcases for a technical and semi-technical audience. ELSNET’s role with respect to these showcases can be passive (i.e. just inclusion as pointers to some external site, or materials downloadable from the ELSNET website), or active (i.e. integration in the ELSNET website, which may require special integration, presentation and adaptation efforts).

In parallel, ELSNET will continue to use e-mail lists (both for the network as a whole and for sub-networks), which remain an efficient means to quickly communicate modest volumes of easy-to-digest information and requests.

Since the beginning of ELSNET, ELSNews (a newsletter written and produced to professional standards) has established itself as a channel of communication keeping members of the widely-scattered European academic and industrial language and speech community abreast of relevant new developments within Europe and outside, and drawing outsiders' attention to the breadth of such activity in Europe. An informal poll taken by the current ELSNET Information Dissemination Task Group co-ordinator when he took on that role in 1998 showed that members of the community saw ELSNews as a valuable resource providing them with a spectrum of information not conveniently available anywhere else. Under the follow-up project the newsletter will continue to be produced (T6) on its present schedule. Per issue 1200 copies are sent to 600 addresses in and outside Europe.

This Newsletter will also be published electronically on the web site.

As an upgrade of our current informal referral service, by which people from within or outside ELSNET send us queries to get hold of specific information, we will operate a quick referral service for those looking for specific expertise or information. The audience will include researchers, providers, integrators, advanced deployers, and educators. 'Naive' end-users fall outside the scope of this activity, but we will of course help them if we can.

The referral service will receive queries via phone, fax, email, web, and will provide pointers to the relevant parties or even bring them into contact.

This service will explicitly include bringing together industrial parties looking for innovative technologies and components, and academics that have potentially commercially exploitable results but who lack the contacts, funds or expertise to patent and exploit them. This will be set up as a proactive service, whereby parties are approached in order to identify their needs and the technologies or research results they would like to share with others.

The knowledge base used by the service will in part be locally available documents and electronic information, but more importantly the collective expertise represented by the network as such. It goes without saying that every member of the network should be willing to make his/her knowledge available to this service.

Best practice documents are another instrument to bring potential developers or deployers into contact with state-of-the-art information. As there exists no systematic repository for such information, ELSNET will create such a facility, where the documents (or the pointers to them) are gathered from existing sources, e.g. projects or project clusters leading to such documentation. It will provide easy access to these data via the web, and add annotations such as those needed to update the information contained in the database.

Proactively, it will identify cases where new actions (updates of existing documents, creation of new best practice documentation) are due, and where possible take steps towards implementing those actions.

ELSNET has, from its very inception, played a very active role in these three areas. They are important in that they constitute the firm basis on which all collaborative activity has to be built. ELSNET’s role is not to build resources, perform evaluation or set standards, but rather to provide a platform where activities can be co-ordinated, information (both from outside and from inside the EU) can be shared, and innovative initiatives can be taken.

At the level of the HLT programme we will collaborate with the upcoming CLASS project in the field of evaluation, in order to disseminate results and experiences gained in CLASS clusters, and to compare these findings with those of others within or outside the HLT programme.

We see a special role for ELSNET as a discussion platform in the context of emerging EU-US collaboration in the field of evaluation, where it is crucial that expertise and experience gained in evaluation programmes on both sides of the Atlantic are shared, as evaluation is not only conceptually difficult, but also very expensive in terms of both linguistic and human resources. We feel that we need US know-how in order to proceed quickly enough while our US colleagues would appreciate our capacity to handle multiple languages.

ELSNET will liaise with other actions such as ISLE, in order to ensure maximum impact of their actions.

The role of ELSNET with respect to resources will be essentially to detect, through the continuous observation of the HLT landscape and its evolution, the need for new types of language resources and new forms of activities in the field of language resources which will support R&D activities, and to promote small, very focused initiatives aiming to:

• better define the needs detected;
• promote/support the creation of small initial innovative models of resources, which imply a methodological or technical "risk";
• explore the feasibility and various forms of organisational activities for resource building, maintenance and sharing, which imply a certain degree of uncertainty;
• give initial support to these types of initiatives, or others that require timeliness.
• stimulate the evolution of the field through its "nursery" function.

These initiatives could later on follow their own independent development path, but the multi-facet collective know-how represented by ELSNET and the relative flexibility and rapidity of decisions have already proved to be very important in the past, e.g. in the development of ELRA (which was designed by the ELSNET resources task-force) or in the organisation of the recent concertation meeting, co-sponsored by ELSNET, between international organisations and national projects working in the field of language resources. This action line, aiming at concertation and synergy between national resources projects will be continued.

ELSNET will also organise specific events, workshops on advanced topics, e.g. in conjunction with major international conferences, in particular conferences in the field of language resources. The language resources task-force will also co-operate in preparing, in collaboration with other actors, the "road map" for the development of language resources, which will be an important part of the ELSNET technology-watch strand of activities.

As for our collaboration with ELRA, it is worth noting that, as it already happened in the past, model resources produced by ELSNET are distributed by ELRA. ELRA's current policy necessarily tends to avoid risk, and concentrates on mature LR, which have or can have an immediate market and users. ELSNET will explore (through its "nursery" functions) and stimulate new developments and offer an advisory service to ELRA.

As for system-level interoperability standards we see no obvious role for ELSNET. When we look at standards at the level of interoperability of language and speech related components a modest role is possible. Although it should be clear that ELSNET is not in a position to develop or impose standards, it will exploit its collaborative structure by liasing with standards making committees and activities in order to organise feedback and dissemination, and by facilitating early application and validation of emerging de facto standards.

In some areas, closer to academia, ELSNET will be able to play a more active role, as it can serve as a platform for exchange of ideas and experimentation, notably in the field of language and speech resources, with a focus of new types of resources, annotation schemes, and tools.

The EC has the right to use and disseminate all intermediate and final results arising from this project, both during its execution and after its completion, according to the provisions stipulated in Annex II.