Title of Invention

SYSTEM FOR AN INTUITIVE, CUSTOMIZABLE, MULTILINGUAL AND RECONFIGURE ABLE AUGMENTATIVE COMMUNICATION .

Abstract

An alternative and augmentative communication system (AAC) and in particular alternative and augmentative communication system adapted for easy and intuitive manner of communication which can be customized and reconfigured and also adapted to operate in multilingual mode. The system basically comprises content developing means; graphical user interface means; and a reconfigurable and multilingual Indian language based natural language generation means. The system and method of communication is adapted for user friendly interface support to cater to requirement of easy access involving different modes of communication based on the users capacity to interact and is directed to a more intuitive and reliable alternative and augmentative communication.

Full Text

FIELD OF THE INVENTION The present invention relates to alternative and augmentative communication system (AAC) and in particular alternative and augmentative communication system adapted for easy and intuitive manner of communication, which can be customized and reconfigured and also adapted to operate in multilingual mode. The system and method of communication of the invention is adapted for user friendly interface support to cater to requirement of easy access involving different modes of communication based on the users capacity to interact and is directed to a more intuitive and reliable alternative and augmentative communication. BACKGROUND OF THE INVENTION The alternative and augmentative communication systems are basically directed to facilitate communication by individuals whose own abilities to communicate are not sufficient enough and need to interact and communicate by use of some support communication system. Usually such systems are adapted to receive inputs from the user and translate the input into an output message, which communicate the intended communication. Such known augmentative support systems include visual displays and synthesized speech. It is always important to develop augmentative communication systems, which could improve upon the rate of communication of successful communication with minimum possible user intervention. Such requirement of alternative and augmentative communication therefore necessitates the development of user interfaces with coding system to generate desired messages. In particular all such devices usually utilize codes supplied by the user to generate desire messages. Therefore the need to code the messages has led to the development of several coding systems and depending upon the user interfaces such variety of codlings usually are known to employ truncated words, words without vowel, contractions or sequences of graphic characters as input codes. Such coding are usually known to involve fixed code system where the user is required to specify a fixed code to generate a specific message alternatively it is also known be variable where the coding usually depend upon factors such as frequency or recency of use or the probable grammatical character of the message. It is also known to have augmentative communication employing databases containing variety of messages. In such cases the user can indicate the selected output message in the form of a string. Importantly also the alternative and augmentative communication system are well known to be a multi disciplinary involving areas of Artificial Intelligence (AI), Natural Languages Processing (NLP), Human Computer Interaction (HQ), Psychology and the like. US6289301 discloses a system and method for frame- based augmentative communication using pre- define lexical slots. US626007 discloses a system and method for frame based augmentative communication having a predefined nearest neighbor association between communication frames. US5956667 describe the system and method for frame based augmentative communication. US6266631 discusses another system and method for frame based augmentative communication having pragmatic parameters and navigational indicators. It would be apparent from the above that these are related to augmentative communication systems involving stored sets of pre defined messages for specific situations and adapted to retrieve them in an efficient manner according to the needs of the user. This is essentially betterment of the stored sentence* communication mode. US5299125 discloses a natural language processing system and method for parsing a plurality of input symbol sequences into syntactically or pragmatically correct word massages which is not sufficiently robust enough for wide language coverage. US5210689 discloses a system and method for automatically selecting among a plurality of input modes. US6005549 disclosed an user interface method and apparatus. It would be apparent from the above that while several variety of alternative and augmentative communication systems are presently in use, given the purpose of such systems there is a constant need for its development to favour wide language coverage and also its reconfigurability for multilinguality. Moreover, considering the variety and nature of incapacity of individuals such systems to favour wide application should preferably involve user friendly access so that user can access and use the system and communicate conveniently and reliably. It is also found that apart from the multi lingual character such systems should preferably have the adaptability were the user can input instructions in one language and generate the communication in another language suited for the purpose. In some situations standard peripherals like keyboard, mouse etc. known for use in such computer based communication system are not sufficient for the user and these also affect the required user friendly characteristics and limit augmentative communications. It is also found that simple coding of communicable messages is not always sufficient and it is experience that there is a need for imparting reconfigurability, intuitiveness and multilingual features in such augmentative communication systems for a wider and more effective and reliable alternative communication system. OBJECTS OF THE INVENTION The basic object of the present invention is directed to provide an alternative and augmentative communication system (AAC) and a method of alternative and augmentative communication and in particular to an alternative and augmentative communication system / method adapted for easy, user friendly application and which is reconfigurable for any selected language. Another object of the present invention is directed to provide an alternative and augmentative communication system/method wherein the input and / or output of every module / sub module can be customized and standardized. It is another object of the present invention to provide an alternative and augmentative communication system adapted to operate in a multilingual mode. Yet further object of the present invention is directed to provide user-friendly characteristics to alternative and augmentative communication systems by favoring options of different modes of communication based on the individuals choice and abilities to communicate keeping in view the reliable alternative and augmentative support required. It is another object of the present invention is to provide for alternative and augmentative communication system/method which would favour icon based natural languages generation and communication. Further object of the present invention to directed to impart features of customizability and intuitiveness in alternative and augmentative communication system for more reliable user friendly end use / application. SUMMARY OF THE INVENTION Thus according to the basic aspect of the present invention there is provided a system for an intuitive, customizable, multilingual and reconfigure able augmentative communication comprising: communication means using icons providing iconic mode of communication mode, virtual keyboard providing keyboard mode of communication and pre-stored messages providing stored message communication mode; user access mechanism; content developing means adapted to carry out content developing using natural language root words associated with icons and sentences/phrases; graphical user interface means for carrying out different modes of communication as stated above and content development; and a reconfigurable and multilingual Indian language based natural language generation means. In accordance with a preferred aspect of the present invention there is provided a system for an intuitive, customizable, multilingual and reconfigureable augmentative communication system comprising: special access mechanism for communication selected from (a) stored sentence mode of communication , (b) keyboard mode of communication wherein the keyboard supports adaptations and predictions in Indian languages and (c) iconic mode of communication; content developing means; font converter means; graphical user interface means; a reconfigurable and multilingual Indian language based natural language generation means. In accordance with yet another preferred aspect of the present invention there is provided a system for an intuitive, customizable, multilingual and reconfigureable augmentative communication system comprising: special access mechanism for communication selected from (a) stored sentence mode of communication , (b) keyboard mode of communication wherein the keyboard supports adaptations and predictions in Indian languages and (c) iconic mode of communication; content developing means; font converter means; graphical user interface means; a reconfigurable and multilingual Indian language based natural language generation means; and text-to speech converter means preferably Bengali/Hindi text -to-speech systhesizer. In the above system of the invention the prediction and adaptation of the keyboard in Indian Languages provide for the following features (i) Prediction: The system is adapted such that when a user is typing a word through the keyboard, after typing in a few characters, the keyboard can predict the whole word. For example, from "app" the keyboard can predict the word "apple". Similarly, from a word, the keyboard can predict the next word and (ii) Adaptation: By adaptation it is meant adapting the prediction module of the system according to the users need. For example, though from "app", multiple words can be predicted, like "apple", "application", "apply" and so on, based on the users preferences, the keyboard will predict only a single word (say "apple"). In accordance with an aspect the natural language generating means Is adapted to generate the said output in another language comprising means to match the SSU of one language (input) with the SSU of another language (output) using multilingual SSU map. The system includes means to match concepts in different languages using multilingual dictionary inputs. For the purpose of content customization the system comprises means to selectively modify anyone or more of the following: a) list of stored data; b) forest data structure along with icon-concept map information; c) lexical knowledge base used by NLG module. Importantly, in the system of the invention the said content development means is adapted for intuitive interaction of the user with a graphical user interface. The said intuitive interaction means comprise an SSU model. in accordance with an aspect the special access mechanism comprise scanning means preferably means for automatic focusing- defocusing of GUI elements, means for auto scrolling and special hardware switches. Such scanning means include block-scanning mechanism along with the row and column scanning mechanism. According to a preferred aspect the said access mechanism is adapted for stored sentence mode of communication and comprises: i) means to provide a list of sentences to the user; and ii) means for the user to select a sentence required to be executed and/or spoken out by a text to speech converter preferably Bengali/Hindi text-to- speech systhesizer. The system involves ITRANS (a standard of representing Indian language text using roman alphabets) for internal representations to store the sentences when said language of communication is vernacular. The user can modulate and pre select the number of sentences to be displayed at a time for users convenience. Also the sentences can be automatically scanned sequentially. A means for visual representations along with the sentences being viewed can also be provided to favour communication. According to another preferred aspect the said access mechanism is adapted for keyboard mode of communication wherein the keyboard supports adaptations and predictions in Indian languages. In such case the system involves a virtual keyboard comprising of alphabets of the selected language, comprising the text involving edit features and displaying the composed text in a display means. According to a yet further preferred aspect the said access mechanism is adapted for carrying iconic mode of communication by the user. For the purpose of such iconic communication the system comprises: means for considering set of icon-text pairs by the user each representing a concept; means for interacting with the system with the question and possible answer based feedback mechanism; means for completing the selection process and outputting system-generated text. In accordance with an aspect of the present invention the means for generation of the natural language comprise: i) converter means to convert the UI file from the ICM to SSU frames; ii) SSU planner means adapted for ordering the unmodified SSU frame; Hi) Value Modifier means, wherein each of the SSU from the frame is extracted and sent to die SSU Modifier for extraction of each QH block present in the SSU and forward the same to the QH Modifier and also extracting the verb from the frame and sending to Verb modifier iv) Means for combining the modified SSU and AV Blocks together to produce a modified SSU frame as the output of frame value modifier; v) Sentence maker means adapted to receive the modified frame as input and produce the final sentence as output from this frame. Advantageously the system comprises two resources comprising (i) a SSU-PP map which lists the pre / post positions for each SSU and (ii) a lexical knowledge database. The QH block Modifier according to an embodiment can comprise (i) qualifier planner adapted to arrange qualifiers inside a QH block using a resource file that encodes knowledge about qualifier ordering in a form of regular expressions (ii) a qualifier modifier adapted to property inflect each qualifier according to a linguistic rule and (iii) a headword modifier adapted to modify the headword according to some linguistic rules stored in a rule base. In accordance with another aspect of the present invention there is provided a method for an intuitive, customizable, multilingual and reconfigure able augmentative communication using the above system comprising: selectively using a key means for activating special access mechanism for communication selected from (a) stored sentence mode of communication, (b) keyboard mode of communication wherein the keyboard supports adaptation and prediction and (c) iconic mode of communication. content developing by providing natural language root words associated with icons and sentences/phrases; using a Semantico-Syntactic Unit based natural language generator for generating natural language sentence output from user input; and font converter to display non-English natural language text. In accordance with a preferred aspect of the present invention there is provided a method for an intuitive, customizable, multilingual and reconfigures ble augmentative communication comprising: Using a special access mechanism for communication selected from (a) stored sentence mode of communication, (b) keyboard mode of communication wherein the keyboard supports adaptations and predictions in Indian languages and (c) iconic mode of communication;content developing ;font converter;and using a graphical user interface and a reconfigurable and multilingual Indian language based natural language generation means. In accordance with yet another preferred aspect of the present invention there is provided a method for an intuitive, customizable, multilingual and reconfigureable augmentative communication comprising: using a special access mechanism for communication selected from (a) stored sentence mode of communication, (b) keyboard mode of communication wherein the keyboard supports adaptations and predictions in Indian languages and (c) iconic mode of communication. content developing ; font converting; using a graphical user interface means , a reconfigurable and multilingual Indian language based natural language generation means; and text-to speech converter means preferably Bengali/Hindi text-to-speech systhesizer. In the above method of the invention the said communication is following anyone or more of (i) stored sentence communication mode, (ii) keyboard communication mode wherein the keyboard supports adaptations and predictions in Indian languages and (iii) Iconic communication mode. Advantageously in the above method before the actual communication is started the user sets the language of communication as well as selects the options whether the special access mechanism is to be activated or not. To generate the said output in another language the SSU of one language (input) is matched with the SSU of another language (output) using multilingual SSU map. The method also uses multilingual dictionary resource for matching of concepts in different languages and communicate in the multilingual mode. The content customization can be advantageously carried out by selectively modifying anyone or more of the following: a) list of stored data; b) forest data structure alongwith icon-concept map information; c) lexical knowledge base used by NLG module. The method effectively makes use of special access mechanism comprising scanning preferably automatic focusing- defocusing of GUI elements, auto scrolling using special hardware switches. Thus the method of the present invention carries out block scanning mechanism in addition to the row and column scanning mechanism. In accordance with one aspect the stored sentence mode of communication comprises: i) considering a list of sentences by the user; ii) selection of a sentence by the user to be executed and/or spoken out by a text to speech converter preferably Bengali/Hindi text-to-speech synthesizer. The said language of communication can be vernacular using ITRANS (a standard of representing Indian language text using roman alphabets) for internal representations to store the sentences. In accordance with another aspect the keyboard mode of communication involving adaptations and predictions in Indian languages comprises: considering of a virtual keyboard by the user comprising of alphabets of the selected language, composing the text involving edit options and displaying the composed text in a display area . In accordance with another aspect the iconic mode of communication used by the user. Comprise: means for considering set of icon-text pairs by the user each representing a concept; ; means for interacting with the system with the question and possible answer based feedback mechanism; means for completing the selection process and out putting system-generated text. Following the above method the output is generated by the user by specifying the exact type affirmative, negative, interrogative, tense aspect and modalities of the desired single output sentence. In the above method the natural language is generated following the steps of : i) the UI file from the ICM is sent to the converter whereby the UI file is converted to SSU frames; ii) forwarding the unmodified SSU frame to the SSU planner to arrange the SSUs in proper order; iii) forwarding the ordered SSU to frame value modifier ,wherein each of the SSU from the frame is extracted and sent to the SSU Modifier for extraction of each QH block present in the SSU and forward the same to the QH Modifier and also extracting the verb from the frame and sending to Verb modifier iv) the modified SSU and AV Blocks are combined together to produce a modified SSU frame which is output of frame value modifier; v) providing the modified frame as input to the sentence maker to produce the final sentence as output from this frame. The said step of conversion of UI file to a SSU comprises use of two resources comprising (i) a SSU-PP frame which lists the pre/post positions for each SSU and (ii) a lexical knowledge database. In particular, for the arrangement of the SSU in proper order the resource used is the knowledge of the relative SSU ranks. Preferably, in the above method the said OH block modification is operated using a (i) qualifier planner to arrange qualifiers inside a QH block using a resource file that encodes knowledge about qualifier ordering in a form of regular expressions (ii) a qualifier modifier to properly inflect each qualifier according to a linguistic rule and (iii) a headword modifier to modify the headword according to some linguistic rules stored in a rule base. BRIEF DESCRIPTION OF THE ACCOMPANYING FIGURES The details of the invention its object and advantages are explained hereunder the greater detail in relation to non limiting exemplary illustrations as per the accompanying figures wherein: Figure 1 illustrates in blocks the various units and operative communication there between in the system of the invention; Figure 2 is segment of the data file containing words; Figure 3 is an illustration of the graphical user interface of the system; Figure 4 is an illustration of a list of stored sentences in non English text (Bengali) that can be used by the system; Figure 5 is a illustration of the virtual keyboard involving adaptations and predictions in Indian languages such as non English (Bengali) v/ith various editing facilities adapted for use in the system; Figure 6 is an illustration of a single sentence output that can be produced by the system from a selected set of icons; Figure 7 illustrates the wide communicability of the system to produce all possible sentences as output (of which 4 is shown here) from a selected set of icons; Figure 8 shows the state transition diagram representing the behavior of GUI; Figure 9 illustrates in greater detail the natural language generator of the system; Figure 10 shows the state transition diagram representing the behavior of special access mechanism; and Figure 11 illustrate the another embodiment of special access switches. Reference is first invited to Figure 1 which illustrates by way of a block representation the features in accordance with a preferred aspect of the system and method for an intuitive, customizable, multilingual and recofigurable alternative and augmentative communication in accordance with the present invention. As represented in said figure the basic essential means used by the system involve the Content Developer (CD), Graphical User Interface (GUI) and the Natural Language Generator (NLG), which are shown shaded. The Content Developer (CD) means is used to populate the system with contents which is adapted to basically provide two types of content (i) natural language root words associated with icons and (ii) sentences / phrases. The root words are selectively adapted to be stored in a hierarchical tree format in a text file as clearly shown in Figure 2. Each record in the data file as shown in said Figure 2 comprise of the following parte Word type > where, Word type indicates whether this is root node or a normal node in the tree. Word category indicates whether the word is a verb / adjective/adverb etc. Tree path indicates the path in the tree starting from root node whose end node is the word. Word is the actual root word. The Content Developer (CD) is also adapted to produce a text file containing the information related to word-icon association. The records in this file is of the form: . Another text file contains the stored sentences / phrases. Using this module, user can customize the system content according to his / her need. In particular for the purpose of customizability the system is adapted such that the content as well as the layout of the interfaces (i.e. the way the content is displayed to the user) can be changed as per the requirement of the user. This is an essential requirement of any AAC system because the content and layouts are user dependent. For example, a set of icons may be relevant to a particular user but the same set can be not so relevant (or even totally irrelevant) to some other user. Similarly, a particular way of displaying the system content may be of interest to a particular user or a group of users with similar requirements but may not be so to some other user (s). The former is related to system content customization whereas the later is related to layout customization (technically called adaptation). The system is adapted such that the Content Developer (CD) operatively interacts with the Graphical User Interface (GUI) module as also shown in Figure 1 for facilitating content customization. For the purpose the system is provided with means for storage of content data including: list of stored sentences (used in SCM); forest data structure along with icon-concept map information (used in ICM);and lexical knowledge base used by NLG module. The CD module is adapted to add or modify the existing content and such adaptation features are inbuilt into the system and doesn't require direct intervention of the user. Another essential aspect of the system is the Graphical User Interface (GUI) means which is adapted to help the user to perform various communication tasks in an easy and intuitive way. In Figure 3, a screen shot of the interface is given. In particular in accordance with an important aspect of the system of the invention the interface is specifically adapted to support three different modes of communication. (i) Communication through pre stored sentence / phrases (ii) Communication through typed messages (with the help of a virtual keyboard) and (Hi) Communication via icon selection. In the first mode of communication, a set of pre-stored sentences / phrases is shown to the user. User can select any one from the list and the system then "speaks out" the selected sentence / phrase using a text to speech synthesizer means. Figure 4 gives a screen shot of the system in this particular mode. As clearly apparent from said figure the SCM (stored sentence mode of communication) is adapted for the following: showing a list of sentences to the user ; if the language of communication is vernacular, then the internal representation used to store the sentences is ITRANS (a standard of representing Indian language text using roman alphabet); to display non-English text, FC is used to convert internal representation to the font of the particular language before displaying it to the user; the system displays a limited number of sentences at a time (10 in the current system), if number of stored sentences exceeds that number, there are options to get next or previous set of sentences; if the SAM module is enabled, these sentences will be automatically scanned sequentially; if a sentence is selected by the user (either using SAM or normally with mouse/ keyboard), the selected sentence will be "spoken out" by the TTS; It is also possible to incorporate icons along with text in this mode. There is an exit option that can be used to exit from this mode of communication. In the second mode, a virtual keyboard involving adaptations and predictions in Indian languages (such as in the user's native language) with various editing facilities is provided to the user. User can "type in" text using this keyboard and can "speak out" the typed text with the help of a text to speech synthesizer. In figure 5, a screen shot of such a virtual keyboard in Bengali is given, along with different editing facilities. In this KCM (keyboard mode of communication) the system is thus adapted for the following: showing a virtual keyboard is shown to the user; the keyboard is provided with alphabet of the said language, a display area to display the composed text and a set of standard editing options that includes {Back Space, Space, Clear, Speak, Exit}; for non-English language communication, FC is used to display keys and composed text in the native language; for vernacular languages, the internal representation that is used to store composed text is ITRANS; using speak option, the composed text can be "spoken out" (using the TTS module). The exit option can be used to exit from this mode of communication. In the third mode, user selects a set of icons from the interface. The system then can form this selected icon set. Depending upon the cognitive ability of the user, this mode can operate in two variants. In one variant, user specifics the type of sentence he/she needs and correspondingly the system outputs a single sentence of that particular type. However specifying the sentence outputs all possible sentences that can be formed out of the selected icon set and the user can choose any one from them. The selected sentence can also be "spoken out" using a text - to - speech synthesizer. Figure 6 and figure 7 shows the two variant of the system working in this particular mode. Figure 8 demonstrates the behavior of the interface in terms of state transition diagram in this mode. An exemplary list of actions based on which a state transition can occur is as follows: An item is selected and the item is a verb; An item is selected and the item is a word; An item is selected and the item is a verb / word category; Item to display next set of icons is selected; Item to display previous set of icons is selected; Item to undo the current selections made so far is selected; Item to go to adverb selection state is selected; Item to go to word qualifier selection state is selected; Item to exit the selection mode and go to output mode is selected. A more detailed illustration is apparent from Figure 8 which shows the various states indicative of: BS (P) → user can select only a verb in this state PS (PC) → user can select a verb category in this state BS (A) → user can select adverb (s) in this state HS (P, PC) → user can select either a verb or a verb category HS (W, WC) → user can select either word (s) or word category BS (W) → user can select only word (s) in this state BS (Q) → user can select word qualifier (s) in this state PS (WC) → user can select a word category in this state C1, C2 → these are control states to determine next state of the system Thus in this ICM (Iconic mode of communication) the system is adapted for : i) Showing a set of icon-text pair is shown to the user; ii) Each icon-text pair represents a concept. For example, the concept of weat" will be displayed to the user as an icon of eating along with the text "eat". iii) The system is verb driven. User has to select a verb first to communicate something. iv) Whenever user selects a verb, the system is adapted to ask a series of questions to the user (each question corresponds to a SSU of the SSU frame of the verb). The SSU frame is a structural model of natural language sentences. Each question is accompanied by a set of possible answers. v) User can select one/more answer (s) of a question or can skip the current question and goes to the next (if any)/previous (if any) question. vi) In case of wrong selection (s), user can undone it. vii) A verb can be qualified by adverbs (ex. moving fast) or a word can be qualified by word qualifiers (ex. a red ball). viii) Once the selection is complete, the system goes to output mode. ix) The output (natural language sentence) can be produced in two different ways. In one case, user specifies the exact type (affirmative/negative/interrogative), tense, aspect, and modality of the desired output sentence. The system accordingly produces a single output sentence. x) In other variant, the system produces all possible output sentences (syntactically different) that can be produced from the user selection. User can choose one from this output set. xi) In the first variant, user can optionally "speak out" the generated sentence whereas in the second case, the system automatically "speaks out" the selected sentence. xii) A concept hierarchy is used to display large number of concepts in a limited display area. For example, large numbers of verbs are organized under verb categories; large numbers of words are organized under word categories etc. xiii) There are options to restart the whole communication process (in ICM) (by using restart option) or to exit this particular mode (using exit option). Importantly, the system of the invention achieves intuitiveness since the GUI module is adapted to work in an interactive manner. This is particularly true in ICM mode of communication where this feature is primarily required. The ICM mode is verb driven. User has to select a verb first to initiate communication. For example, to indicate "I ate an apple yesterday", user will first select the main verb "eat". Then the system will pose a series of questions to the user along with possible set of answers for each question. User can select one/more answer (s) from the set. For example, to construct the above sentence, once user selects the verb "eat", the system will ask the question "who" along with possible answers {I, You, He, She, Father etc}. An icon will accompany each of these possibilities. User will select "I" from this set. Similarly, user will select "apple" for "what" and "yesterday" for "when". After this selection is complete, the GUI module will invoke the NLG module to construct the appropriate sentence. This easy to understand question-answering technique of communication is intuitive and incorporated in the system of the invention. The adaptability and reconfigurability of the system of the invention in such ionic mode of communication is further illustrated in greater detail in relation to the components of natural language generator (NLG) of the present system as per accompanying Figure 9. In particular a sequential summary of the working and functionality of the various modules is given below. The UI file from ICM is sent to the converter (UC) adapted to convert the UI file to a SSU frame. Two resource bases are operatively linked for such conversion. A SSU-PP map that lists the pre / post positions for each SSU, and a lexical knowledge base containing the lexical category, person, number, gender information of lexical items (verbs, words, qualifiers). The SSU frame generated by the converter is unmodified. It contains SSUs that are not in proper order. The value of each SSU is also not modified and contains root words only. Furthermore, the qualifiers in individual QH blocks (constituents of a SSU) are not properly ordered. The verb is also not inflected. This unmodified SSU frame is then sent to the SSU planner (SP) that arranges the SSU in proper order (the order in which they actually appear in a sentence). The resource used here is the knowledge of relative SSU ranks (the relative rank of each SSU with respect to other SSUs in a language). The output of SP is a SSU frame with ordered SSUs but unmodified values. The verb is also unmodified. It is then sent to the frame value modifier (FVM) to modify all these values. FVM consists of two independent components, the SSU modifier (SM) and the verb modifier (VM). In FVM, each SSU from the frame is extracted and sent to the SM. Similarly the FVM extracts the verb from the frame and sends it to the VM. SM consists of QH_block modifier (QM). SM extracts each QH_block present in the SSU and sends it to the QM to get it modified. The QM consists of three components. They are (1) a qualifier planner (QP) that arranges qualifiers inside a QH_block. QP uses a resource file that encodes knowledge about qualifier ordering in the form of regular expressions, (2) a qualifier modifier that properly inflects each qualifier according to some linguistic rule and (3) a headword modifier that modifies the headword according to some linguistic rules. The linguistic rules are stored in the rule base. The output of a QM is a modified QH_block. All these modified QH_blocks are composed together according to certain linguistic rules by SM to form a modified SSU. The VM modifies the AV_block with the help of a rule base. All these modified SSUs and AV_block are combined together to produce a modified SSU frame. This is the output of FVM. This modified frame is then taken as input by the sentence maker and produces final sentence as output from this frame. Importantly, the range of sentences that can be produced using the system of the invention m fairly large in number (using the SSU model). To reduce the resulting cognitive load, the GUI is designed to work In a question-answering mode (using the semantics of SSU). Moreover, using the question-answering mode of ICM along with the SSU model of natural language sentences in the system of the invention, the limitations of the existing systems to produce sentences having only at most three components, namely agent, action and patient (theme) can be avoided. For example, sentences like "John ate an apple". However, these types of sentences cover a very limited domain in any natural language, even if the said domain is restricted to grammatically simple sentences only. The way these sentences are produced by the existing system is not particularly suitable to produce even simple sentences with many components. For example, sentences like "John, Alex and Mary are going to the movie today with Mary's uncle in a white ambassador". If the NLG techniques that are used in the existing system is used to produce this type of sentences from the input set {John, Mary, Alex, go, movie, Mary, uncle, white, ambassador}, then either a large amount of information is required to produce the said sentence or there will be many possibilities from which one possibility has to be selected. Using SSU model of the invention, sentences that are even more complicated than the one cited above can be generated easily with little cognitive load on the user. Thus the SSU model increases the flexibility in the communication process by giving the user power to generate a large number of sentences in an easy way. In accordance with another aspect of the invention the system is provided with a versatile, reconflgurable and intuitive Natural Languages Generator (NLG) means. This is adapted to generate natural language sentence (s) from the selected set of icons. The design of each SSU is either a noun or pre / postpositional phrases along with some semantic information embedded into it. Any natural language simple sentences can be viewed as a collection of such SSU's along with one / more verb phrases (consisting of a verb with adverbs).. Furthermore, any complex sentence can be analyzed as a composition of a number of such simple SSU frames. The semantic information is used to perform the ordering of different SSU components as well as ordering the SSUs in a sentence. The semantic information also helps to some extent to the design of an intuitive user interface. The natural language generator takes the user selection as its input, produces a SSU frame from that input, performs certain linguistic task on the frame and finally produces the semantically and syntactically correct natural language text as output. Figure 9 further shows the architecture of the NLG module. Importantly also in accordance with another aspect the system is adapted for the multilingual mode of operation in Iconic communication mode (ICM). The multilingual characteristics in the system of the invention is achieved by way of: The NLG module of the system is completely re-configurable. So system in different languages (Bengali, Hindi and other Indian languages as well as English) can be made easily by re-configuring the NLG module. User can provide input in one language and can get the output in some other language. However, for this to happen, the NLG module incorporates two additional resources: A multilingual SSU map that maps the SSU of one language to the corresponding SSU of another language .A multilingual dictionary that contains mapping of concepts in different languages (e.g., the concept "Eating" will have an entry in the multilingual dictionary as "Eat" in English, "khAoYA" in Bengali, "khAnA" in Hindi etc.). The modified ICM in that case is illustrated in Figure 9 (the gray colored boxes indicate the resources). Advantageously, the NLG in the present system is highly modular where the I/O of each module along with the resources they use is clearly specified (the basic architecture is shown in the next figure). The formats of the resources (the gray colored boxes) are very easy to understand and change. This complete standardization of the architecture and resource formats is the key to language re-configurability. By changing a few sub- modules and/or some resource files will result in a NLG modules for a new language. The SSU model also helps in this process by facilitating identification and standardization of the resources and making them easy to change, even for a non- technical person. Regarding TTS, addition of Indian language TTS (for Bengali and Hindi language communication) is a novel feature in such systems and were not done before. Advantageously further the system of the invention is re-configurable. That means all the modules or sub-modules are specifically adapted to be replaced with new ones (with same functionality) that conform to the I / O standard of the specific module / sub- module. For example, by changing few sub-modules of the NLG module (namely QH_block modifier, Verb Modifier), NLG for three different languages (Bengali, Hindi, English) can be developed. Similarly, by changing the output component of the ICM sub- module of the GUI module, two different GUI versions of the system can be developed. In accordance wit a preferred aspect of the present invention the system comprises of Special Access Mechanism (SAM) adapted for people with severe neuro-motor disabilities. This segment of user cannot use normal peripheral devices like keyboard involving adaptations and predictions in Indian languages, mouse etc due to their physical disability. Using the SAM module, users can access the system with special purpose access devices. For the purpose the system is provided with special access switches. These switches can be used in different modes like single switch two switch etc. In single switch mode, the on-screen items are automatically scanned one after another and the switch is used to select a particular item while it is being scanned. In two-switch mode, one switch is used to "shift" the scan from one item to another and the other switch is used to select a particular item. Figure 10 illustrates the mechanism through a state transition diagrams. User can customize the scanning speed and number of switches according to his / her need. As represented in said figure R is indicative of → register switch is "clicked"; S is indicative of → shift switch is "clicked"; F is indicative of → finished input selection and RST is indicative of → restart selection process In accordance with a preferred aspect the SAM module consists of means for scanning such as automatic focusing-defocusing of GUI elements, auto-scrolling and means for special access switches. In particular in accordance with a preferred aspect the special access mechanism includes: i) A keyboard involving adaptations and predictions in Indian languages adapted with features for block scanning mechanism in addition to the row and column scanning mechanism; ii) Auto-scrolling mechanism adapted to display large number of concepts in a limited display area; iii) Special access switches. In accordance with another preferred aspect of the system of the invention there is provided Font converter means adapted to display non-English natural language text. In case of Indian languages, the system internally works with a standard known as ITRANS. The font converter module is used to convert from ITRANS to respective language fonts. In case of English, this module is not required. In accordance with yet another aspect the system of the invention involves a Text to Speech Synthesizer means adapted to "speak out" whatever text user wants to communicate. For Bengali and Hindi, indigenously developed TT5 systems are being used to "speak out" the text. In the system of the invention the CD operation is the first to be performed (at least once) in the system. However, due to the feature of customizability and personalization with respect to varied need of the users, later CD operations are also permitted. Before the actual communication process starts, user can set the language of communication as well as whether to start SAM or not. Depending on those choices, the system automatically invokes FC and SAM modules (if required). In the GUI process, user can use any of the three modes of communications by selecting the appropriate option. If invoked, SAM, FC and GUI operates in parallel. In ICM, once a user finishes selection of icons (thus constructing an UI file), he / she can invoke NLG module to get a textual output. Optionally, user can invoke the "ITS module to get a spoken output. In SCM, selection of an item automatically invokes the TTS whereas in KCM, the TTS is optional. In accordance with another aspect of the present invention, FC, TTS, and NLG can be used in other applications also. For example, in a machine translation'system, the FC component can be used to display generated target language text if the text is not in English, whereas the NLG can be used as the target language generator. One of the many applications of the TTS is as an integral component in a screen-reader system for visually handicapped. It is thus possible by way of the present invention to provide a system for alternative and augmentative communication, which can be easily reconfigured to develop systems for any particular language. The input / output of every module / sub module is adaptable for standardization such as for the Bengali, Hindi and English version of the systems; Based on the present preferred language adaptability of the system for Bengali, Hindi and English the following language pairs, namely Bengali-Hindi, Hindi-English, English- Bengali, Bengali-English, Hindi-Bengali, and English-Hindi can be generated. The method can be extended to other languages that are structurally similar to any of the three languages, namely Bengali, Hindi or English. The system is also adapted to operate in multilingual mode. This is particularly relevant in case of third mode, of communication, namely communication through icons. In this mode, user selects icons. Each icon has a text associated with it that conveys the meaning. The system produces natural language sentence(s) based on the selection. However, user can select icons in a particular language (meaning the text with each icon will be displayed in that language) and the system can produce output in some other language. In this way, the system can be used in a multilingual environment also. Moreover, the system is also customizable and intuitive to serve as a user friendly and a more reliable communicable media. WE CLAIM: 1. A system for an intuitive, customizable, multilingual and reconfigure able augmentative communication comprising: communication means using icons providing iconic mode of communication mode, virtual keyboard providing keyboard mode of communication and pre-stored messages providing stored message communication mode; user access mechanism; content developing means adapted to carry out content developing using natural language root words associated with icons and sentences/phrases; graphical user interface means for carrying out different modes of communication as stated above and content development; and a reconfigurable and multilingual Indian language based natural language generation means. 2. A system as claimed in claim l comprising: special access mechanism and said graphical user interface adapted to support communication selected from (a) stored sentence mode of communication, (b) keyboard mode of communication wherein the keyboard supports adaptation and prediction in Indian languages and (c) iconic mode of communication; and font converter means for font conversion to display non-English text. 3. A system as claimed in anyone of claims 1 or 2 comprising: text-to speech converter means with Bengali/Hindi text-to-speech synthesizer. 4. A system as claimed in anyone of claims 1 to 3 comprising means to selectively set the language of communication as well as activate the special access mechanism. 5. A system as claimed in anyone of claims 1 to 4 wherein natural language generating means comprises Semantico-Syntactic Units (SSU) to provide for textual output. 6. A system as claimed in anyone of claims l to 5 comprising reconfigurable natural language generating means to generate different languages comprising of SSU frame based natural language generator. 7. A system as claimed in claim 6 wherein the natural language generating means is adapted to receive input in one language and generate output in another language. 8. A system as claimed in claim 7 wherein said natural language generating means comprises multilingual SSU map provided to generate the said output in another language by matching the SSU of one language (input) with the SSU of another language (output). 9. A system as claimed in anyone of claims 7 or 8 comprising means to match concepts in different languages using multilingual dictionary inputs. 10. A system as claimed in anyone of claims 1 to 9 comprising means to carry out content customization and / or layout customization. 11. A system as claimed in claim 10 wherein content customization means comprise means to selectively modify anyone or more of the following: a) list of stored data; b) forest data structure along with icon-concept map information; c) lexical knowledge base used by NLG module. 12. A system as claimed in anyone of claims l to 11 wherein said content development means is adapted for intuitive interaction of the user with a graphical user interface in the iconic communication mode. 13. A system as claimed in anyone of claims 1 to 12 wherein said intuitive interaction means comprise an SSU based model of natural language sentences. 14. A system as claimed in anyone of claims 1 to 13 comprising access mechanism comprising scanning which further comprises means for automatic or user-directed focusing- defocusing of GUI elements, means for auto scrolling and special access switches, said special access switches also comprising a. a switch interface connected to a computer port (serial/parallel/USB) b. one/two access switches connected to the switch interface. 15. A system as claimed in claim 14 comprising block scanning mechanism along with the row and column scanning mechanism. 16. A system as claimed in anyone of claims 1 to 15 wherein said access mechanism is adapted for stored sentence mode of communication and comprises: i) means to scan a list of sentences; ii) means for the user to select a sentence from the list and be spoken out by a Bengali/Hindi text to speech converter. 17. A system as claimed in claim 16 comprising ITRANS (a standard of representing Indian language text using roman alphabets) based internal representations to store the sentences when said language of communication is vernacular (Bengali/Hindi). 18. A system as claimed in anyone of claims 1 to 17 comprising font converters to convert internal representations in ITRANS to the font of the particular language (Bengali/Hindi) before displaying the same. 19. A system as claimed in anyone of claims1 to 18 wherein said special access mechanism is adapted such that the sentences are scanned sequentially where the scanning is either automatic or user directed. 20. A system as claimed in anyone of claims 1 to 19 comprising means for iconic representations along with the texts being viewed in the stored sentence mode of communication. 21. A system as claimed in claim 1 to 20 wherein said access mechanism is adapted for a virtual keyboard used in the keyboard mode of communication that have support for adaptations and predictions in Indian languages, said virtual keyboard also comprising of alpha-numeric characters of the selected language, the text editing features and means for displaying the composed text. 22. A system as claimed in anyone of claims 1 to 21 wherein said special access mechanism is adapted for carrying iconic mode of communication by tfte user. 23. A system as claimed in claim 22 comprising: means for providing set of icon-text pairs to the user each representing a concept; means for user interacting with the system with the question and possible answer based interaction mechanism; means for completing the selection process and outputting system- generated text and speech (using text-to-speech converters). 24. A system as claimed in anyone of claims 22 or 23 comprising user-specific output generation by providing user-interactive input means for specifying the desired single sentence output. 25. A system as claimed in anyone of claims1 to 24 wherein the means for generation of the natural language sentence comprise: a) converter means to convert the user inputs stored in a file (UI) to SSU frames; An SSU frame comprises an AV (Adverb-Verb) block and a set of unmodified SSUs each of which further comprises of a set of QH (Qualifier- Headword) blocks. b) SSU Planner means adapted for ordering the SSUs in the unmodified SSU frame; c) Frame Value Modifier means comprising means for, i. Means for extracting each SSU from the SSU frame and sent to SSU Modifier which extracts each QH block from the SSU and sends to the QH Modifier which modifies the QH block. ii. Means for extracting AV block from the SSU frame and sending it to the Verb Modifier which modifies the AV block, iii. Means for combining the modified SSU and AV Blocks together to produce a modified SSU frame as the output of the frame value modifier; d) Sentence Maker means adapted to receive the modified frame resulted from the actions of (b) and (c) above as input and produce the final sentence as output from this frame. 26. A system as claimed in anyone of claims1 to 25 comprising two resources comprising (i) a SSU-PP map which lists the pre / post positions for each SSU and (ii) a lexical knowledge database. 27. A system as claimed in anyone of claims1 to 26 wherein said QH block Modifier comprises (i) qualifier planner adapted to arrange qualifiers inside a QH block using a resource file that encodes knowledge about qualifier ordering in the form of regular expressions (ii) a qualifier modifier adapted to properly inflect each qualifier according to linguistic rules and (iii) a headword modifier adapted to modify the headword according to linguistic rules stored in a rule base. 28. A method for an intuitive, customizable, multilingual and reconfigure able augmentative communication using the system as claimed in anyone of claims 1 to 27 comprising: selectively using a key means for activating special access mechanism for communication selected from (a) stored sentence mode of communication, (b) keyboard mode of communication wherein the keyboard supports adaptation and prediction and (c) iconic mode of communication. content developing by providing natural language root words associated with icons and sentences/phrases; using a Semantico-Syntactic Unit based natural language generator for generating natural language sentence output from user input; and font converter to display non-English natural language text. 29. A method as claimed in claim 28 comprising Bengali/Hindi text-to-speech conversion for speech-impaired users. 30. A method as claimed in anyone of claims 28 to 29 wherein before the actual communication is started setting the language of communication as well as selecting the options of the spedal access mechanism is to be activated. 31. A method as claimed in anyone of claims 28 to 30 wherein the natural language is generated in different languages by reconfiguring the natural language generator. 32. A method as claimed in anyone of claims 28 to 31 wherein the natural language is generated by providing input in one language and generating output in another language. 33. A method as claimed in claim 32 wherein to generate the said output in another language the SSU of one language (input) is matched with the SSU of another language (output) using multilingual SSU map. 34. A method as claimed in anyone of claims 32 or 33 comprising using multilingual dictionary resource for matching of concepts in different languages. 35. A method as claimed in anyone of claims 28 to 34 comprising carrying out content customization and / or layout customization. 36. A method as claimed in claim 35 wherein content customization is carried out by selectively modifying anyone or more of the following: a) list of stored data; b) forest data structure alongwith icon-concept map information; c) lexical knowledge base used by NLG module. 37. A method as claimed in anyone of claims 28 to 36 wherein said iconic communication is carried out involving intuitive interaction of the user with a graphical user interface containing icons. 38. A method as claimed in anyone of claims 28 to 37 wherein said intuitive interaction comprise using an SSU based model of natural language. 39. A method as claimed in anyone of claims 28 to 38 comprising using special access mechanism comprising scanning comprising automatic or user- directed focusing-defocusing of GUI elements and auto scrolling using special hardware access switches. 40. A method as claimed in anyone of claims 28 to 39 comprising carrying out block scanning mechanism in addition to the row and column scanning mechanism. 41. A method as claimed in anyone of claims 28 to 40 wherein said stored sentence mode of communication comprises: i) selection of a sentence by the user and be spoken out by a Bengali/Hindi text to speech converter. 42. A method as claimed in claim 41 of using ITRANS (a standard of representing Indian language text using roman alphabets) as internal representations to store the sentences when the language of communication is vernacular. 43. A method as claimed in anyone of claims 28 to 42 comprising using font converters to convert internal representations in ITRANS to the font of the particular language before displaying the same when the language of communication is non English. 44. A method as claimed in anyone of claims 28 to 43 wherein when using the spedal access mechanism the sentences are scanned sequentially either automatically or in a user-directed way. 45. A method as claimed in anyone of claims 28 to 45 comprising activating an exit button after the communication is over. 46. A method as claimed in anyone of claims 28 to 45 comprising methods for carrying out communication using a virtual keyboard that supports adaptation and prediction. 47. A method as claimed in claim 46 wherein said communication using virtual keyboard further comprises: considering of a virtual keyboard by the user comprising of alphabets of the selected language, composing the text using the alphabet keys and edit options and viewing the composed text in a display area. 48. A method as claimed in anyone of claims 28 to 47 for iconic communication mode comprising: considering set of icon-text pairs by the user each representing a concept; interacting with the system using a question and possible answer based interaction mechanism; completing the selection process and outputting system-generated text. 49. A method as claimed in anyone of claims 28 to 48 wherein the natural language is generated following the steps of: i) the file containing the user selections (UI) made by the user from the iconic communication mode Interface is sent to the converter whereby the UI file is converted to a unmodified SSU frame; ii) forwarding the unmodified SSU frame to the SSU planner to arrange the SSUs in proper order; iii) forwarding the ordered SSU frame to the frame value modifier, wherein each of the SSU from the frame is extracted and sent to the SSU Modifier for extraction of each QH block present in the SSU and forward the same to the QH Modifier and also extracting the verb from the frame and sending to the Verb modifier iv) the modified SSU and AV Blocks are combined together to produce a modified SSU frame which is output of the frame value modifier; v) providing the modified frame as input to the sentence maker to produce the final sentence as output from this frame. 50. A method as claimed in claim 49 wherein said step of conversion of UI file to a SSU comprises use of two resources comprising (i) a SSU-PP mapping which lists the pre / post positions for each SSU and (ii) a lexical knowledge database. 51. A method as claimed in anyone of claims 49 or 50 wherein for the arrangement of the SSU in proper order the resource used is the knowledge of the relative SSU ranks. 52. A method as claimed in anyone of claims 49 to 51 wherein said OH block modification is performed using (i) a qualifier planner to arrange qualifiers inside a QH block using a resource file that encodes knowledge about qualifier ordering in a form of regular expressions (ii) a qualifier modifier to properly inflect each qualifier according to linguistic rules and (iii) a headword modifier to modify the headword according to linguistic rules stored in a rule base. 53. A system for an intuitive, customizable, multilingual and reconfigure able augmentative communication and a method of carrying out augmentative communication using said system substantially as herein described and illustrated with reference to the accompanying figures. An alternative and augmentative communication system (AAC) and in particular alternative and augmentative communication system adapted for easy and intuitive manner of communication which can be customized and reconfigured and also adapted to operate in multilingual mode. The system basically comprises content developing means; graphical user interface means; and a reconfigurable and multilingual Indian language based natural language generation means. The system and method of communication is adapted for user friendly interface support to cater to requirement of easy access involving different modes of communication based on the users capacity to interact and is directed to a more intuitive and reliable alternative and augmentative communication.

Documents:

463-KOL-2004-(22-03-2012)-CORRESPONDENCE.pdf

463-KOL-2004-(22-03-2012)-FORM-27.pdf

463-KOL-2004-CORRESPONDENCE.pdf

463-KOL-2004-FORM 27.pdf

463-kol-2004-granted-abstract.pdf

463-kol-2004-granted-claims.pdf

463-kol-2004-granted-correspondence.pdf

463-kol-2004-granted-description (complete).pdf

463-kol-2004-granted-drawings.pdf

463-kol-2004-granted-examination report.pdf

463-kol-2004-granted-form 1.pdf

463-kol-2004-granted-form 18.pdf

463-kol-2004-granted-form 2.pdf

463-kol-2004-granted-form 3.pdf

463-kol-2004-granted-pa.pdf

463-kol-2004-granted-reply to examination report.pdf

463-kol-2004-granted-specification.pdf