Title of Invention

A METHOD AND AN APPARATUS FOR PROCESSING INFORMATION

Abstract In an information processing apparatus, a processing instruction is obtained from a displayed web page in response to an operation of a button or a mark provided on th4 page, the obtained processing instruction is transferred to a processing unit which is able to execute the processing instruction, and the transferred processing instruction is executed by the processing unit which is also in the processing apparatus or in a server connected to the apparatus via a network.
Full Text

METHOD AND APPARATUS FOR EXECUTING AN INSTRUCTION IN A WEB PAGE AND FOR MANAGING IMPORTED OR EXPORTED DATA
CLAIM FOR PRIORITY
This application claims priority from Application Nos, 2002-067868,2002-111801, 2002-141403, and 2002-198009 filed on March 13,2002, April 15, 2002, May 16, 2002 and July 5, 2002 respectively in JAPAN.
FIELD
This invention relates to an information processing method and apparatus, and more particularly a method and apparatus for executing a processing instruction during a web page is displayed.
BACKGROUND
A conventional personal computer can access the Internet and display a web page using browser software. Currently, some web sites, provide a mechanism which can print, send the contents, or store the links of web pages. However, they do not allow the saving of the contents of the web page, processing of the web page like annotating the contents, sending annotated contents, or extracting only required data from the displayed page.
Accordingly, there is a need for a method and apparatus that resolves the above-mentioned problems.
SUMMARY
According to one embodiment, the present invention relates to an information processing method comprising the steps of obtaining a processing instruction from a displayed web page; and transferring the obtained processing instruction to a processing unit which is able to execute the processing instruction.
According to another embodiment, the present invention relates to an information processing method comprising the steps of obtaining a processing instruction from a displayed web page; and executing the obtained processing instruction.

According to still another embodiment, the present invention relates to an information processing apparatus comprising obtaining means for obtaining a processing instruction from a displayed web page; and transfer means for transferring the obtained processing instruction to a processing unit which is able to execute the processing instruction.
According to yet another embodiment, the present invention relates to an information processing apparatus comprising obtaining means for obtaining a processing instruction from a displayed web page; and execution means for executing the obtained processing instruction.
According to a further embodiment, the present invention relates to an information processing system in which a plurality of client terminals are connected to a server via a network, wherein each of said client terminals comprising obtaining means for obtaining a processing instruction from a displayed web page; and transfer means for transferring the obtained processing instruction to a processing unit which is able to execute the processing instruction.
According to a further embodiment, the present invention relates to a computer-executable program for controlling a computer to perform information processing, said program comprising codes for causing the computer to perform obtaining a processing instruction from a displayed web page; and transferring the obtained processing instruction to a processing unit which is able to execute the processing instruction.
According to a further embodiment, the present invention relates to a computer-executable program for controlling a computer to perform information processing, said program comprising codes for causing the computer to perform obtaining a processing instruction from a displayed web page; and executing the obtained processing instruction.
According to still another embodiment, the present invention relates to a data processing method comprising the steps of obtaining data displayed by a browser; importing data without displaying the data; and storing and managing the obtained data and the imported data.
According to another embodiment, the present invention relates to a data processing method comprising the steps of obtaining data displayed by a browser; storing and managing the obtained data; and exporting the managed data.

According to still another embodiment, the present invention relates to a data processing apparatus comprising obtaining means for obtaining data displayed by a browser; importing means for importing data without displaying the data; and management means for storing and managing the obtained data and the imported data.
According to yet another embodiment, the present invention relates to a data processing apparatus comprising obtaining means for obtaining data displayed by a browser; management means for storing and managing the obtained data; and exporting means for exporting the managed data.
According to a further embodiment, the present invention relates to a computer-executable program for controlling a computer to perform data processing, said program comprising codes for causing the computer to perform the steps of obtaining data displayed by a browser; importing data without displaying the data; and storing and managing the obtained data and the imported data.
According to a further embodiment, the present invention relates to a computer-executable program for controlling a computer to perform data processing, said program comprising codes for causing the computer to perform the steps of obtaining data displayed by a browser; storing and managing the obtained data; and exporting the managed data.
Other features and advantages of the present invention will be apparent from the following description taken in conjunction with the accompanying drawings, in which like reference characters designate the same or similar parts throughout the figures thereof.
BRIEF DESCRIPTION OF THE FIGURES
The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.
Fig. 1 is a block diagram illustrating the hardware configuration according to an embodiment of the present invention.
Fig. 2 is a flowchart of the procedural steps of an information processing system to process a web page.

Fig. 3 shows an example of the web page to be processed by the information processing system embodying the present invention.
Fig. 4 is a flowchart of the procedural steps of Instruction Fetcher of this
embodiment.
Fig. 5 is a flowchart of the procedural steps of Instruction Processor of this
embodiment.
Fig. 6 is a flowchart of the procedural steps of Instruction Processor of this embodiment.
Fig. 7 shows an example user interface of step S602 of this embodiment.
Fig. 8 is a flowchart of the procedural steps of Instruction Execution of this embodiment.
Fig. 9 is a flowchart of the procedural steps of Saving the displayed web page to a Filing System.
Fig. 10 shows example KPT System architecture for the information processing system embodying the current invention.
Fig. 11 is a flowchart of the procedural steps of Saving the displayed web page to KPT System.
Fig. 12 is a flowchart of the procedural steps of Getlnmit.
Fig. 13 shows an example web page along with the save user interface of the KPT System.
Fig. 14 shows an example web page along with the user interface directly embedded onto the web page.
Fig. 15 is a flowchart of the procedural steps of Getlnput, wherein part of the parameters are obtained by the embodiment.
Fig. 16 shows an example web page along with the save user interface directly embedded onto the web page.
Fig. 17 shows another example of web page along with the save user interface directly embedded onto the web page.
Fig. 18 is a flowchart of the procedural steps of Send Instruction Processor.

Fig. 19 shows an example web page along with the ^eiifcl user interface directly embedded onto the web page.
Fig. 20 shows an example web page along with the send user interface of the KPT System.
Fig. 21 shows an example web page along with the embedded send user interface.
Fig. 22 shows an example web page along with the annotate user interface of the KPT System.
Fig. 23 is a flowchart of the procedural steps of annotate instruction processor of this embodiment.
Fig. 24 shows an example web page along with the embedded annotate user interface.
Fig. 25 shows another example web page along with tl}e embedded annotate user interface for highlighting.
Fig. 26 shows an example web page along with the information extraction user interface of the KPT System.
Fig. 27 is a flowchart of the procedural steps of information extraction processor of this embodiment.
Fig. 28 shows an example web page along with the embedded information extraction user interface.
Fig. 29 shows an example User Interface of Retrieve, sorted by time or session order.
Fig. 30 shows an example User Interface of Retrieve, Get All Links UI sorted by Domains.
Fig. 31 shows an example User Interface of Get All Links UI sorted by Keywords.
Fig. 32 shows example User Interface of Retrieve.
Fig. 33 shows a Client-Server type implementation.
Fig. 34 shows an Application Service Provider (ASP) type implementation.
Fig. 35 is a flowchart of the procedural steps of Instruction Processor.

Fig. 36 shows an example user interface of the login screen displayed.
Fig. 37 is a flowchart of the procedural steps of Instruction Processor for Client Server or ASP type of architecture.
Fig. 38 shows an example web page after the save has been performed.
Fig. 39 shows an example User Interface of Retrieve, sorted by time or session order.
Fig. 40 is a flowchart of the procedural steps to determine if an instruction can be executed.
Fig. 41 shows an example user interface of the error message displayed to the user, when the save cannot be executed.
Fig. 42 is another flowchart of the procedural steps to determine if an instruction can be executed.
Fig. 43 shows an example web page along with the user interface after the save has been performed.
Fig. 44 shows an example display to explain how the limitation becomes applicable.
Fig. 45 shows an example user interface of the error message displayed to the user, when the save cannot be executed.
Fig. 46 shows an example web page along with the processing toolset directly embedded on the web page.
Fig. 47 shows an example web page along with the processing toolset for getting details of save.
Fig. 48 shows an example web page along with the annotation toolset for directly annotating on the web page being displayed.
Fig. 49 shows another example of web page along with1 the highlight annotation toolset for annotating on the web page being displayed.
Fig. 50 shows an example web page along with Extraction tool set to extract relevant information directly from the web page.
Fig. 51 shows another example User Interface of Retrieve.
Fig. 52 shows an example of other processes performed by this embodiment.

Fig. 53 shows an example knowledge base to keep track of the processing performed for a given page.
Fig. 54 shows an example of toolset to search for keywords within the
displayed page.
! Fig. 55 shows alternative system architecture for the embodiment.
Fig. 56 shows an example web page along with the 'Keep this' button and anchor displayed as part of the page.
Fig. 57 shows an example JavaScript program embedded within the web page
shown in Fig. 56.
' Fig. 58 is a flowchart of the procedural steps of Broker Agent.
Fig. 59 is a flowchart of the procedural steps of CheckForKPTSystem.
Fig. 60 is a flowchart of the main procedural steps.
Fig. 61 is a flowchart of INITIALIZE procedure.
Fig. 62 is a flowchart of MAIN-PROCESSOR.
Fig. 63 is a flowchart of the procedural steps of UserAction.
Fig. 64 is a flowchart of the procedural steps of CheckExisting.
Fig. 65 is a flowchart of the procedural steps of Annotate.
Fig. 66 is a flowchart of the procedural steps of Extrac .
Fig. 61 is a flowchart of the procedural steps of Mgmt.
Fig. 68 is a flowchart of the procedural steps of ExecuteAction.
Fig. 69 is a flowchart of the procedural steps of SaveContents.
Fig. 70 is a flowchart of the procedural steps of WebFetch.
Fig. 71 is a flowchart of the procedural steps of SendContents.
Fig. 72 is a flowchart of the procedural steps of SaveFileContents.
Fig. 73 is a flowchart of the procedural steps of FillKptAction.
Fig. 74 is a flowchart of the procedural steps of TERMINATE.
Figs. 75 and 76 show examples of the knowledge structures in the knowledge base.
Figs. 77 and 78 show examples content of the knowledge base.
Fig. 79 is a flowchart of the procedural steps to process the PRINT command.

Fig. 80 is an example of Select Printer UI.
Fig. 81 is an example print output of the web page.
Fig. 82 is an example user interface within the web page to obtain the various parameters for PRINT,
Fig. 83 is an example print output of multiple web pages laid out onto a single page.
Fig. 84 shows another system architecture for this embodiment, when applied to Information Kiosk.
Fig. 85 shows an example web page displayed on the Information Kiosk along with commands embedded within the web page.
Fig. 86 is a flowchart of the procedural steps to process the commands on the Information Kiosk.
Fig. 87 shows an example UI of user settings.
Fig. 88 shows an example UI of user settings for makirjg a CD for the information selected by the user.
Fig. 89 shows an example UI of for making the CD for the information collected by the user.
Fig. 90 shows another example of web page related to historic information with 'Get this!'button.
Fig. 91 shows another example of web page related to product information with 'Get this!'button.
Fig. 92 shows an example user settings for printed matter delivery.
Fig. 93 shows an example information database for this embodiment.
Fig. 94 shows another system architecture for the implementation of a standalone Information Kiosk for this embodiment.
Fig. 95 shows another system architecture for the implementation of a networked Information Kiosk for this embodiment.
Fig. 96 shows example architecture, when multiple clients access the embodiment.

Fig. 97 is a flowchart of the procedural steps of INITIALIZE for the system shown in Fig. 96.
Fig. 98 is an example Login UI for the step S9704.
Fig. 99 is a flowchart of the procedural steps of TERMINATE for the system shown in Fig. 96.
Fig. 100 shows another functional block diagram, of information processing system embodying the present invention.
Fig. 101 is a flowchart of the main procedural steps.
Fig. 102 is a flowchart of INITIALIZE procedure.
Fig 103 is a flowchart of the procedural steps of Mgmi
Fig. 104 is a flowchart of the procedural steps of ShowSession.
Fig. 105 is a flowchart of the procedural steps of ShowLinks.
Fig. 106 is a flowchart of KPTIMPORT.
Fig. 107 is a flowchart of the procedural steps of ImportData.
Fig. 108 is a flowchart of the procedural steps of IndexData.
Fig. 109 is a flowchart of KPTEXPORT.
Fig. 110 is a flowchart of the procedural steps of ExportData.
Fig. 111 is an example of Import UI.
Fig. 112 is an example user interface of browsing files to import.
Fig. 113 is an example of Show all Links sorted by Domain.
Fig. 114 is an example of Show all Links sorted by Organizations.
Fig. 115 is an example of Show Sessions.
Fig. 116 is an example of Show all Links sorted by Keywords.
Fig. 117 is another example of Show Sessions, in which the imported conference sessions are shown.
Fig. 118 is another example of Show all Links sorted by Keywords, for imported conference information.
Fig. 119 is a warning dialog to prompt the user to insert the CD containing the full proceedings of the conference.

Fig. 120 is another example of Show all Links sorted by Keywords, for imported conference information.
Fig. 121 is another example of Show all Links sorted by Organizations, for imported conference information.
Fig. 122 is another example of Show all Links, sorted t>y Table of Contents.
Fig. 123 is another example of Show Sessions, in which the conference information and other web information saved are shown in separate nodes.
Fig. 124 is another example of Show all Links, sorted by Keywords, in which the conference information and other web information saved are shown in separate nodes.
Fig. 125 is another example of Show all Links, first sorted by Organization and then by Authors below each node.
Fig. 126 is another example of Show all Links, sorted by Authors and Organization.
Fig. 127 is an example user interface of AutoFetch List.
Fig. 128 is an example user interface of specifying a new auto fetch instruction.
Fig. 129 is an example user interface for specifying the| output report to be generated.
Fig. 130 is an example user interface to specify the information sources to look for.
Fig. 131 is an example of Show Sessions of the auto fetch results.
Fig. 132 is another example of Show Sessions of the auto fetch results in a graphical format.
Fig 133 is another example of Show Sessions of the auto fetch results in a graphical format.
Fig 134,135 and 136 are examples of Show Sessions of the imported information.
Fig. 137 is another example of Get all Links for imported association members.
Fig. 138 is an example user interface of Export.
Fig. 139 is an example user interface to specify the information to be exported.
Fig. 140 is an example user interface to specify somp p^arameters for export.

Fig. 141 is an example user interface to specify the format of the information to be exported.
Fig. 142 is an example of exported data in a browser independent format.
Fig. 143 is another example of exported data, in which the information can be sorted by the user.
Fig. 144 is another example user interface to specify the information to be exported.
Fig. 145 is an example output of the exported address book data.
Fig. 146 is an example output of the exported address book data in CSV format.
Fig. 147 is an example output of the exported address book data in Text format.
Fig. 148 is an example output of the exported address book data in HTML format.
Fig. 149 is example information as seen in a browser of the exported address book data.
Fig. 150 is an example of viewing the published information from this embodiment.
Fig. 151 is a flowchart of the procedural steps of ShowLinks.
Fig. 152 is a flowchart of the procedural steps of CreateTree.
Fig. 153 is a flowchart of the procedural steps of SortTreeView.
Fig. 154 is a flowchart of the procedural steps of ShowTreeView.
Fig. 155 is a flowchart of the procedural steps of ProcessLeafAction.
Fig. 156 is a flowchart of the procedural steps of ProcessNodeAction.
Fig. 157 is another example of user interface for Import,
Fig. 158 is an example user interface to specify the emails to be imported.
Fig. 159 is another flowchart of the procedural steps of:SaveContents.
Fig. 160 is another flowchart of the procedural steps WebFetch.
Fig. 161 is an example of Get all Links sorted by Sender for the imported email information.
Fig. 162 is another example of Get all Links sorted by Sender for the imported email information.

Fig. 163 is an example of Show Sessions for the imported email information. Fig. 164 is an example of Get all Links sorted by Name for the imported
documents.
Fig. 165 is another example of Get all Links sorted by Name for the imported
documents.
Fig. 166 is a flowchart of the procedural steps of ImportData. Fig. 167 is a flowchart of the procedural steps of GetType. Fig. 168 is a flowchart of the procedural steps of Setlncjlex.
DETAILED DESCRIPTION
A preferred embodiment of the present invention will now be described in detail with reference to the accompanying drawings.
Fig.l is a block diagram illustrating the hardware configuration according to an embodiment of the present invention. In this figure, a central processing unit (CPU) 101 is operative to perform operations for various processing and make a logical decision or the like and further controls each composing element connected to a bus 107.
A RAM 102 temporarily stores variables and intermediate data generated during the processing. A program from an external source may be loaded into the RAM 102. A ROM 103 is used to store programs, which correspond to individual flowcharts that will be described later and which are to be executed by the CPU 101, and fixeid data.
A keyboard (KB) 104 is used for inputting data and ani(instruction by a user. A mouse or other input devices may be used with the keyboard 104. Display 105 displays data and a hard disk drive (HDD) stores data of a database, a program, andthe like.
The bus 107 is used to transfer an address signal indicating a composing element to be controlled by the CPU 101, a control signal used for controlling each composing element and data to be exchanged between the composing equipments.
Fig. 2 is a flowchart of the procedural steps of an information processing system to process a web page. Web Page Contents Fetcher 201 is used for fetching the information from the server for allowing the information browsing of the web. Web Page Displayer 202 is used for displaying the actual contents of the web information. Instruction

Fetcher 203 is an input means to used to get the instruction for processing the contents of the web page. Instruction Processor 204 is for processing and executing the instructions fetched by the Instruction Fetcher.
Fig. 3 shows an example of the web page to be processed by the information processing system embodying the present invention. As can be seen from the figure, using the "Keep This", the web page currently being displayed can be saved. Similarly, using the "Print This" button, to print the page contents, "Send This" button to send the contents. Similarly, any other functionality can be added to the "XYZ" button, which while browsing lets the user perform various actions on the web page. For example, the user can annotate, highlight the contents of the web page, save or file the annotated web page, send the information to other people or extract only relevant information, retrieve saved information, translate the whole contents or part of contents, get a abstract of key points of the page and so on. The button referred to in this embodiment can also be implemented in other ways like for example an anchor or a link or a menu or any other type of display control.
Fig. 4 is a flowchart of Instruction Fetcher 203 of this embodiment. In step S401, ACTION is set the user action performed by the user. In step S402 a check is made to determine if the ACTION is null or not. If so, the process ends otherwise, a check is made in step S403 to determine if the ACTION is an INSTRUCTION. If not, the browser in step S405 processes the ACTION and the process ends. The normal user actions like navigating to a different page or link, changing the character-set or encoding of the displayed page, changing the font size etc. are typical actions, which are performed by the browser. Otherwise, the instruction for processing the web page, for annotating, saving, processing the currently displayed web page is fetched and set as INSTRUCTION in step S404 and the process ends.
Fig. 5 is a flowchart of the procedural steps of Instruction Processor 204 of this embodiment. The Instruction Processor can be of two types. In the first case, the INSTRUCTION procured is directly processed by the program embedded onto the web page in step S501. For example, annotating onto a web page or data extraction from a web page can be directly performed. In the second case, the Instruction is forwarded in step S502 to the actual Instruction executor, which executes the instruction in step S503. For example, saving or sending the web page contents can be processed in either of the abqve two ways.

Instructions can be broadly classified into two types. Instruction Type A, whose processing is explained in Fig. 6, like Save, Send,... which do not directly affect the contents of the page being displayed and are final actions which can be performed by either the client or on the server. Instruction Type B like Annotate, Extract,..., whose processing explained in Fig. 8 affect the contents of the web page being displayed and are not the final resultant actions. For example, after annotating the page, the user needs to perform some other action like Save, Send, Print etc. Similarly after Extracting, the user needs to save, send, export to address book, print or dump to file etc. Also the Instructions of Type B have to be performed on the client, since they directly affect or have to performed on the contents of the web page.
Fig. 6 is a flowchart of the procedural steps of Forward Instruction S502 of this embodiment. A check is made in step S601 to determine if the instruction execution module for this Instruction exists. If so, the process proceeds to step S605, otherwise, step S602 is executed, wherein the download and install screens are shown to the user, an example of which is shown in Fig. 7 and proceeds to step S603, to determine if the Instruction executor needs to be obtained or downloaded now. If not, an error message is displayed in step S604 and the process ends. Otherwise step S605 is executed, wherein a check is made to determine if the instruction can be processed. If so the process proceeds to step S607, otherwise an error message is displayed in step S606 and the process ends. In step S607, a check is made to determine if the input parameters are required to be obtained. If so, the process proceeds to step S608, wherein Getlnput is executed whose details will be explained later. In either case step S609 is executed, wherein the instruction is passed on to the appropriate instruction executor. In step S610, a check is made to determine if the server needs to be informed. If so step S611 is executed wherein, a message is sent to the server to inform that the page has been processed by the client or the user and if a knowledge base exists in the client it is also updated. This step is performed to collect a log of actions performed l! y various users and can
also be used to generate statistics for e.g., most popular i.e., most sav^d, most sent, etc. In
i
either case the process ends. In step S602, the message can be to download and start the installation process or just a message to inform the user that the program does not exist and has to be downloaded and installed, at which point the user can either download from a website, install using a CD-ROM etc.

Fig 7. is an example user interface of S602. A security warning screen is displayed to assure the user of the authenticity of the component being installed. If user chooses not to install, an error message will be displayed.
Fig. 8 is a flowchart of the procedural steps of Execute Instruction S501 of this embodiment. A check is made in step S801 to determine if the instruction can be processed. If not, an error message is displayed and the process ends. Otherwise, step S803 is executed, wherein, a check is made to determine if the input parameters are required to be obtained. If so, the process proceeds to step S804, wherein Getlnput is executed whose details will be explained later. In either case step S805 is executed, wherein the actual instruction is executed. In step S806, a check is made to determine if the server needs to be informed. If so step S807 is executed wherein, a message is sent to the server to inform that the page has been processed by the client or the user and if a knowledge base exists in the client it is also updated.
Fig. 9 is a flowchart of the procedural steps of Saving the displayed web page to a Filing System. In step S901, a check is made to determine if the file name can be decided without user intervention. If so, the file name is decided in step S902, otherwise step S903 is executed wherein the file name is obtained from the user. In either case in step S904, the actual save to the filing system is performed and the process ends.
Fig. 10 shows example KPT System architecture information processing system
embodying the current invention. The information on the Internet 100 J is browsed using
multiple browsers 1002A, 1002B simultaneously and as explained in this embodiment. KPT
System 1003 handles and processes them separately, interacts and acts; as a controlling system
i as explained in detail in this embodiment to Browser 1002. Knowledge Base Management
1004, is the management of knowledge accessed/stored from/to the Database 1005.
Fig. 11 is a flowchart of the procedural steps of Saving the displayed web page
to KPT System. Steps SI 101 to SI 104 are exactly same as S801 to S804 of Fig. 8. In step
SI 105, the parameters obtained are passed along with the Instruction to the KPT System. The
KPT System executes the forwarded instruction in step SI 106 and in step SI 107, a check is
made to determine if the server needs to be informed. If so, step S1108 is executed wherein, a
message is sent to the server to inform that the page has been processed by the client or the
user and if a knowledge base exists in the client it is also updated. For example, it is not

necessary for the KPT System to have a user interface and hence for example, when the user presses 'Keep This, the KPT System, without any more user intervention, saves the contents of the web page.
Fig. 12 is a flowchart of the procedural steps of S608, S804 Getlnput. A check is made in step S1201, to determine if KPT System has user interface to get the input parameters from the user or otherwise. If so, request or message is sent to KPT System to display UI in S1202, for example Fig. 13 and the parameters obtained from the KPT System in step SI203. Otherwise, the input parameter user interface is displayed directly within the web page, by the associated function in step SI204, for example Fig. 14 and the actual input obtained from the user in step SI205 and the process ends.
Fig. 13 shows an example web page along with the save user interface displayed by the KPT System in step SI202.
Fig. 14 shows an example web page along with the user interface directly embedded onto the web page, displayed in step SI204.
Fig. 15 is a flowchart of the procedural steps of Getlnput, wherein part of the parameters are obtained by the embodiment. Getlnput can also be implemented in other ways. For example, the associated function can first get either the complete input or partial input and then request KPT System to display appropriate user interface. In step SI 501, the input parameter user interface is displayed and the input obtained from the user in step SI502. A check is made in step SI 503 to determine if the user input values. If no, the KPT system is asked to display the user interface in step S1504, otherwise, a check is made in step S1505 to determine if more details are to be input, if not the process returns, otherwise step SI 506 is executed wherein the KPT System is asked to display the specific user interface based on the parameters input by the user. In step SI507, the parameters are obtained and the process returns.
Fig. 16 shows an example web page along with the save user interface directly embedded onto the web page
When user presses, 'Keep This', the user is prompted t > select the type of Save e.g., Quick Save, Save, Hold etc... in the shown example, user has selected Save, hence Save

user interface is presented by KPT System. If QuickSave had been selected no user interface would have been shown by KPT system and the process wtfUld have just saved the page.
Fig. 17 shows another example of web page of Fig. 16 in which case each of the functionality can be put into separate buttons or anchors. For example, if Keep Page and Keep Link can be presented to user as two different buttons.
Fig. 18 is a flowchart of the procedural steps of Send Instruction Processor, which is similar to Fig. 6. A check is made in step SI 801 to determine if conventional means are to be used for sending the information. Conventional means imply, browser or the messaging system will display the user interface, construct the email message, by getting the email recipients, subject and other details, before the message is sent. But in conventional means only send a link, or the page contents without the embedded in ages is allowed. If conventional means are to be used, a check is made in step S1811 to determine if more than conventional data, implying either link only or the page contents without embedded images is to be sent. If only conventional data is to be sent then process proceeds to step SI 808 and passes the data to browser or the messaging system. If more than conventional data has to be sent i.e., for example, embedded images, the input parameter need to be obtained and process proceeds to step SI 806. If conventional means are not be used then, step SI 802 is executed wherein, a check is made in step SI 802 to determine if the instruction execution module for this Instruction exists. If so, the process proceeds to step SI 804, otherwise, step SI803 is executed, wherein the download and install screens are shown to the user, an example of which is shown in Fig. 7 and proceeds to step SI 804, to determine if the Instruction executor needs to be obtained or downloaded now. If not, an error message is displayed in step SI 805 and the process ends. In step SI 806, a check is made to determine if the input parameters are required to be obtained. If so, the process proceeds to step SI807, wherein Getlnput is executed whose details were explained in earlier, the details of what to send, i.e., send only the link, or send the top level page i.e., without embedded images or send the contents along with the embedded images etc... are obtained in SI 807, an example user interface of this is shown in Fig. 20. In either case step SI 808 is executed, wherein the instruction is passed on to the appropriate instruction executor. In step SI 809 a check is made to determine if the server needs to be informed. If so step SI 810 is executed wherein, a message is sent to the server to

inform that the page has been processed by the client or the user and if a knowledge base exists in the client it is also updated.
Fig. 19 shows an example web page along with the send user interface directly embedded onto the web page. As can be seen from the figure, send by conventional means or otherwise are implemented as two separate options i.e., buttons or anchors - "Send This" and "Send with Keepoint" respectively.
Fig. 20 shows an example web page along with the senid user interface of the KPT System.
Fig. 21 shows an example web page along with the embedded send user interface. As can be seen from the figure the user interface is displayed by the button itself or within web page. The details of what to send, i.e., send only the link, or send the top level page i.e., without embedded images or send the contents along with the embedded images etc... are obtained before executing the actual send.
Fig. 22 shows an example web page along with the annotate user interface of the KPT System.
Fig. 23 is a flowchart of the procedural steps of annotate instruction processor of this embodiment. As can be seen from the figure, in step S2301, the user interface to allow the user to annotate the page, is displayed. In step S2302, annotations performed by the user are obtained. In step S2303, the embodiment, incorporates the annotations performed by the user into the page's HTML being displayed and proceeds to step S23CJ4, where the modified HTML page is displayed by the browser. A check is made in step S2305 to determine if the user has finished with the annotation process. If so, the process returns otherwise, the process moves back to S2302 to allow the user to perform more annotations on the page.
Fig. 24 shows an example web page along with the embedded annotate user interface for directly annotating on the web page being displayed. As can be seen in the figure, the user has added a note '[My Comments - Ancient Indian Language]' directly onto the web page next to the word 'Sanskrit' using the 'Add' functionality embedded directly on the web page. The user can also change the text color, highlight portions of text or delete unnecessary text.

Fig. 25 shows another example web page along with the embedded annotate
user interface for highlighting. As can be seen from this figure, part of the text 'source of pride
i to the developers' has been highlighted using the 'highlight' facility of the toolset. The
annotations are directly embedded into the web page as standard HTML tags and no
proprietary mechanism is required to achieve the annotations, which is beneficial to the user,
since the user can send the annotated page by email to a recipient, who can view the annotated
page using a standard web browser.
Fig. 26 shows an example web page along with the information extraction user interface of the KPT System. As can be seen from the figure, the user has extracted the telephone number, '045-913-9820' and the arrow indicates that the user is extracting the number '045-913-9821' as fax number, by just dragging and dropping the same to the fax field of user interface.
Fig. 27 is a flowchart of the procedural steps of information extraction processor of this embodiment. As can be seen from the figure, in step S2701, the user interface which allows the user to extract data from the page is shown, an example of which is shown in figure 28. In step S2702, relevant information or data extracted by theluser are obtained. In step S2703, a check is made to determine if the user has finished with the processing of the data extraction. If not, the process moves back to S2702, otherwise, a check is made in step S2704 to determine if the extracted information needs to be saved. If so, step S2705 is executed, wherein the extracted information is stored either on the client or on the server. In either case, the process returns.
Fig. 28 shows an example web page along with the embedded information extraction user interface. As explained before information extraction is not a final action and hence after extraction the user needs to decided what he or she would like to do with the extracted data. e,g., save, print of export to address book etc. As can be seen from the figure, it is possible to embed this onto pages which have addresses or phone numbers and by clicking on the 'Extract' the user interface with the fields to be extracted are sh|own.
Fig. 29 shows an example User Interface of Retrieve, spited by time or session order. As can be seen from the figure, the stored links or pages are sorted based on domain and listed to enable to the user to easily browse through the stored information.

Fig. 30 shows an example User Interface of Retrieve, Get All Links UI sorted by Domains.
Fig. 31 shows an example User Interface of Get All Links UI sorted by Keywords. As can be seen in Figure 13, the web pages have embedded keywords to enable the search engine and other web spiders to classify the document. This is used by to enable easily retrieval of the stored information.
Fig. 32 shows example User Interface of Retrieve. As can be seen from the figure, various parameters can be input for the search
Fig. 33 shows a Client-Server type implementation. The figure illustrates an example Client-Server type implementation. Multiple clients 3301 have KPT CI system 3302, and communicate with Server 3303. Server 3303 is equipped with KPjT SI System 3304, knowledge base management 3305 and Database 3306. Server 3303 can also communicate with other servers to access their information stored in database, via knowledge base management.
Fig. 34 shows an Application Service Provider (ASP) type implementation. The figure illustrates another possible architecture implementation of the KPT System. Multiple clients 3401, using only a standard web browser, communicate to the web server 3403. The web server 3403 is equipped with KPT S2 System, knowledge base management 3405 and Database 3406. The client and server can be either on the Internet or on the Intranet. This architecture is a typical example of Application Service Provider (ASP) type of implementation of the KPT System.
Fig. 35 is a flowchart of the procedural steps of Instruction Processor 204 for a
typical client server or ASP type of implementation. As can be seen from the figure, a check is
i
made in step S3501 to determine if the INSTRUCTION is null or not If so, the process ends, otherwise the process goes to step S3 502, wherein a check is made to determine if the user is already logged in. If so, the process moves to step S3503, Processlnstruction (Instruction) as was explained in detail in Fig. 37 and the process ends. Otherwise, step S3504 is executed, wherein the user is shown the login screen to logon to the server, be it server of client server or server of ASP or other client of the distributed client systems. In step S3505, a check is made

to determine if the login was successful, if so the process moves to step S3503, otherwise an error message is displayed in S3 5 06 and the process ends.
Fig. 36 shows an example user interface of the login screen displayed by this embodiment in step S3504.
Fig. 37 is a flowchart of the procedural steps of Instruction Processor for Client Server or ASP type of architecture. A check is made in step S3701 to 'determine if the instruction can be processed. If so the process proceeds to step S3703, otherwise an error message is displayed in step S3702 and the process ends, In step S3703, a check is made to determine if the input parameters are required to be obtained. If so, the process proceeds to step S3704, wherein Getlnput is executed whose details were explained eralier. In either case step S3705 is executed, wherein the instruction is passed on to the appropriate instruction executor on the server. In step S3706, a check is made to determine if the server needs to be informed. If so step S3707 is executed wherein, a message is sent to the server to inform that the page has been processed by the client or the user.
Fig. 38 shows an example web page after the save has been performed. After the user uses the 'Keep This' button on the web page to save the web page onto the server, the page is refreshed with the message 'This Page has been saved for 30 days till Feb 11th, 2002' to inform the user that the save was successful.
Fig. 39 shows an example User Interface of Retrieve, sorted by time or session
order for a typical client server or ASP type of implementation System.
Thus it can be seen that this embodiment can also be used in traditional client-server architecture ASP type of implementation and it could be easily seen that it be similarly used with various other different types of architectures.
Fig. 40 is a flowchart of the procedural steps to determine if an instruction can be executed. In steps S605, S801 or S3701, an example flow chart for the check to determine if the save instruction can be performed is shown in this figure. It is possible io use a special version with certain limits, for example, the special version can be limited to save a total of only 100 links or pages and/or limit the space consumed in saving the information to 10MB etc. A check is first made in step S4001 to determine, if certain limits of the special versions have been exceeded, for example, the special version can be limited to save a total of only 100

links or pages and/or limit the space consumed on the server in saving the information to 5MB etc. If the limits have been exceeded an error message is displayed in step S4002 and the process returns false. If not, a check is made in step S4003 to determine if any expiry time limit is to be set on the information being saved. For example, it is possible to set the expiry time limit to say 15 days or one month, so that the KPT System automatically deletes the information after the expiry period. If so, the time limit is set in step S4004. In either case, the process returns true. The idea behind the special or limiting the features of the system, is to provide the above limited version for free and in order for the user to use all the features without any hassles or limitations, the user needs to purchase or procure the full unlimited version.
Fig. 41 shows an example user interface of the error message displayed to the user, when the save cannot be executed in step S4002.
Fig. 42 is another flowchart of the procedural steps to determine if an instruction can be executed. A check is first made in step S4201 to determine if the information currently being displayed can be saved. If not, an error message is displayed in step S4202 and the process returns false. The special version can be constrained to save only limited information, for example, only web pages of a particular web site or a group of web sites or to only sites having the 'Keep This' or 'Keepoint Toolset'. Otherwise, a check is made in step S4203 to determine, if certain limits of the special versions have been exceeded, for example, the special version can be limited to save a total of only 100 links or pages and/or limit the space consumed in saving the information to 10MB etc. If the limits have been exceeded an error message is displayed in step S4204 and the process returns false. If not, a check is made in step S4205 to determine if any expiry time limit is to be set on the information being saved. For example, it is possible to set the expiry time limit to say 15 days or one month, so that the KPT System automatically deletes the information after the expiry period. If so, the time limit is set in step S4206. In either case, the process returns true.
Fig. 43 shows an example web page along with the use,r interface after the save has been performed.
Fig. 44 shows an example display to explain how the limitation becomes applicable. The special version can be constrained to save only limited information, for example, only web pages of a particular web site or a group of web sites or to only sites having

the 'Keep This'. After the 'Keep This5 has been pressed by the user, the KPT System can display the Save UL Thus, in Fig 44, the user can try to save the page by using the 'Save Page' of the KPT System. It is possible to limit this operation in a Special Version, if the page or link is not permitted to be saved, a error message as shown in Fig. 45 will be displayed.
Fig. 45 shows an example user interface of the error message displayed to the user, when the save cannot be executed.
Fig. 46 shows an example web page along with the processing toolset directly embedded on the web page. As can be seen from the figure, instead of individual buttons or anchors, the necessary processing buttons or anchors can be grouped together and presented to the user in form a toolset. The main advantage of presenting this in one set, is the web
i
developer or information provider can easily embed the same onto their web pages.
Fig. 47 shows an example web page along with the processing toolset for getting details of save. As can be seen from the figure, the user for example, can input an identifier in 'Keep As' field, and also specify the validity or the expiry of the information being stored.
Fig. 48 shows an example web page along with the annotation toolset for directly annotating on the web page being displayed. As can be seen in the figure, the user has added a note c[My Comments - Ancient Indian Language]' directly onto the web page next to the word 'Sanskrit' using the Toolset embedded directly on the web page.
Fig. 49 shows another example of web page along with the highlight annotation toolset for annotating on the web page being displayed.
Fig. 50 shows an example web page along with Extraction tool set to extract relevant information directly from the web page. As can be seen from' the figure, the user has extracted the 'Phone' and also assigned 'Keep As' and 'Name' fields. As can be seen from the figure, it is possible to embed the toolset onto pages which have addresses or phone numbers and by clicking on the 'Extract' the user interface with the fields to be extracted are shown.
Fig. 51 shows another example User Interface of Retrieve. When the user presses the 'Retrieve' button on the toolset, a user interface, example of which is shown in this figure, is displayed to allow the user to specify various parameters for searching the information.

Fig. 52 shows an example of other processes performed by this embodiment. As can be seen from the figure, for example, the user can decide to view the 'Most Popular' i.e., the content that was saved most by the users across the ASP System or 'Most Sent', i.e., the content that was sent to friends or other people after viewing by the users of the ASP or client server system. The actual count or statistics are collected in step! S611 or S807 or SI 108
etc..., wherein a message is sent by this embodiment to the server thatj the process has been
i
preformed by the client and the knowledge base is also updated, an example of which is shown in figure 53. Hence, the server side can maintain the information of the number of times; a page has been saved, sent, annotated upon etc.
Fig. 53 shows an example knowledge base to keep track of the processing performed for a given link on a per day basis. It is obvious that the data can be collected individually as a log, i.e. each specific entry or only per URL etc...
Fig. 54 shows an example of a toolset to search for keywords within the displayed page.
Fig. 55 shows alternative system architecture for the embodiment. As can be seen from the figure, the command or request obtained in the browser is forwarded to the KPT System which actually performs the actions via the Broker Agent 5501. The Broker Agent could be for example an Active X Control or a Java Program embedded within the web page.
Fig. 56 shows an example web page along with the 'Ke ep this' button and anchor displayed as part of the page. As can be seen from the figure, tne page contents can be saved by either clicking on 'Keep this' button or clicking on the anchor 'Keep This with Keepoint'.
Fig. 57 shows an example JavaScript program embedded within the web page shown in Fig. 56. When the user clicks either on the button or on the anchor in Fig. 56, a JavaScript program embedded within the web page is executed, to actually embed and hence create the ActiveX control as shown in the figure.
Fig. 58 is a flowchart of the procedural steps of Broker Agent. A check is made in step S5801 using CheckForKPTSystem, details of which are explained in Fig. 59. If the function returns "No", then the process ends. Otherwise, in step S5802, a request is sent to KPT System to display the user interface. In step S5803, a new session is created and in step

S5804, a new task is created. In step S5805, the required KPTAction end KPTDocument knowledge structures are created. In step S5806, the keywords, URL and other information are obtained from the browser. A check is made in step S5807 to determine, if any previous information for the current URL already exists in the knowledge base. If so, the previous information is obtained in step S5808. In either case, the process moves to step S5809, wherein the HTML text is obtained from the browser, along with other information and the KPTAction and KPTDocument structures are filled with the obtained information. In step S5810, the filled KPTAction and KPTDocument structures are passed on to the KPT System along with the request to process the user command.
Fig. 59 is a flowchart of the procedural steps of CheckForKPTSystem. A check is made in step S5901 to determine if KPT System exists in the client machine i.e., if KPT System is installed on the machine. If not, step S5902 is executed wherein a message is displayed to the user and a detailed information - explaining the need to download and install the KPT System along with instructions and system requirements is shown in step S5903 and the process returns false. Otherwise, in step S5904, the version information of KPT System is obtained and a check is made in step S5905 to determine if the KPT System can execute the given command. If so, the process return true, otherwise a message is displayed to the user in S5906 and detailed information requesting the user to either upgrade the KPT System manually or prompt the user for automatic upgrade etc... is performed in step S5907 and the process returns false.
Fig. 60 is a flowchart of the main procedural steps of KPT System. In step S6001, initialization processes to connect to the Internet are executed. In step S6002, main function processing browsing, annotating, saving etc. of this embodiment is performed. In step S6003, terminate or clean-up processing is executed.
Fig. 61 is a flowchart of INITIALIZE procedure of step S6001. In step S6101 a check is made to determine if the browser needs to be instantiated or not. If browser is not instantiated, it is instantiated in step S6102. In step S6103, a new session is created. In step S6104, the knowledge base is updated. A check is made in step S6105!to determine if this is a trial version i.e., a special version that allows all the features to be used and explored before actually purchasing a product. If so, a check is made in step S6107 to determine if the trial

period has expired. For example, a 30 day or 1 week trial period can be set. If not the step S6106 is executed where in the main user interface of the embodiment is displayed and the process ends. Otherwise, step S6108 is executed, wherein the main user interface is displayed in disabled mode to prevent the user from actually using the main functionality of the system and the process ends.
Fig. 62 is a flowchart of MAIN-PROCESSOR of step S6002. In step S6201, a check is made to determine if the browser was instantiated or not. If so, a new session is created in step S6202 and the process proceeds to step S6205, wherein the knowledge base is updated. If not, a check is made in step S6203 to determine if the browser w°is terminated or ended. If so, the session associated with the browser is ended and proceeds to step S6205. If not, a check is made in step S6206 to determine if an action was performed to end the system. If so, all the current tasks are terminated in step S6207 and the process returns.
If not, a check is made in step S6208 to determine if the user is navigating to a new URL. If so, a check is made in step S6209 to confirm with the user that the current task should be terminated. If not, the process proceeds to step S6210, where the navigation is aborted and the process continues to step S6201. If the current task is to be ended in step S6209, step S6211 is executed wherein, the previous task is terminated and then a new task is created. In step S6212, the knowledge structures KPTAction and KPTDocument are created.
In step S6213, the URL and the keywords are obtained from the Browser. A check is made in step S6214 to determine if the URL data already exists in the knowledge base. If so, all the existing data for the current URL is procured from the knowledge base in step S6215 and moves to step S6216, where a check is made to determine if it is a RetrievedURL i.e., the user is trying to view the contents of an already stored page. If so, step S6217 is executed to get the RetrieveUI message and control goes to S6218. If URL data does not already exist in step S6214, step S6218 is executed to display the keywords, other acquired data from browser like the URL, page title etc... and other existing data if any from the knowledge base like Keep As, validity period etc... and the process proceeds to step S6201.
In step S6208, if the user is not moving to a new URL, a check is made in step S6219 to determine if any SystemTask ActL needs to be executed. If so, step S6222 ExecuteAction (ActL) is executed and the control moves to step S6205 to update the

knowledge base. If not, a check is made in step S6220 to determine if any User Operation was performed. If not, step S6205 is executed, otherwise in step S6221, the HTML text is obtained from the browser and the KPT Action and KPTDocument structures created in step S6212 are updated and ExecuteAction(ActL) for the UserAction is executed in s);ep S6222 and the process moves to step. S6205 to update the knowledge base.
Fig. 63 is a flowchart of the procedural steps of S6220 jUserAction. A check is first made in step S6301 to set Act equal to the User operation performed by user and to determine if Act is equal to NULL. If so, the process returns false. If it is not NULL, the process proceeds to step S6302, a check is made to determine whether Act is Create New. If so, the process proceeds to step S6304. If not, the process proceeds to step S6303, a check is made to determine whether Act is Quick Save. If so, the process proceeds to step S6304. If not, a check is made in step S6305 to determine whether Act is Automatic Save. If so, the process proceeds to step S6304. If not, a check is made in step S6306 to determine whether Act is Save. If Act is Save, Save UI, an example of which is shown in Fig. 13, is displayed in step S6307 and proceeds to step S6304. If not, a check is made in step S6308 to determine if the Act is Hold. If so, Hold UI is displayed in step S6309 and proceeds to step S6304. In step S6304, a check is made to determine if the information being saved 01? held is already exists using CheckExisting, which is explained in detail later and if so, retuj^is true, otherwise return false.
In step S6308, if Act is not Hold, a check is made in step S6310 to determine if the Act is Send. If so, Send UI, an example of which is shown in Fig. 20, is displayed in step S6311 and the recipients (To, CC) information, Subject, Contents and/or the like are obtained from the user in step S6312 and proceeds to step S6320. If not, a check is made in step S6313 to determine if Act is Annotate. If so, Annotate UI, an example of which is shown in Fig. 22 is displayed in step S6314 and the actual Annotations executed using Annotate, which is explained in detail later, in step S6315 and process returns true. If not, a check is made in step
56316 to determine if Act is Extract.
If so, Extract UI, an example of which is shown in Fig. 26, is displayed in step
56317 and the actual Extract information executed using Extract, which is explained in detail

later, in step S6318 and process returns true. If not, Mgmts which is explained in detail later is executed in step S6319 and process returns false.
Fig. 64 is a flowchart of the procedural steps of S6304 CheckExisting to check if the information already exists in the knowledge base or not. In step S6401, the values of Keep As, Validity Range etc... are either obtained from the user or from the system settings. In step S6402, a check is made to determine whether the URL already exists. If URL does not exist, the process proceeds to step S6403 where ModifyStatus is set to SaveAsNewAction is set to and returns true. If URL exists, a check is made in step S6404 to determine if the information needs to be over-written (i.e., update or modify the previous information). This is done either by asking the user, whether he would like to overwrite the existing information, save as new do not save or based on the user settings. If so, in step S6405, ModifyStatus is set to OverWriteExisting and the process returns true. If not, a check is made in step S6406 to determine if the information needs to be saved as new, (i.e., without modifying the existing information, save the current information as a separate entity). If so, ModifyStatus is set to SaveAsNewAction and the process returns true. If not, the process returns false.
Fig. 65 is a flowchart of the procedural steps of S315 Annotate and Fig. 22 shows an example user interface for Annotating a web page currently being browsed. In step S6501, the user-performed operation is set to Act. A check is made in step S6502 to determine if Act is Add Note. If so, the user specified memo/note is added to the selected place in step S6503 and goes to step S6504. If not, a check is made in step S6506 to determine if the Act is Footnote. If so, the footnote number is created based on a sequence and the footnote number is added to the selected place in the page and the actual contents of the footnote are added to the end of the page in form of a table and proceeds to step S6504. The notes added to the page are added based on user settings, to set the "annotation demarkers", "default annotation text", the color of annotation etc. The annotations are added as standard HTML tags. If Act is not Footnote in step S6506, a check is made in step S6508 to determine if the Act is Highlight. If
• i
so, a Font tag is added around the selected text with the background color
set to the selected color in step S6509 and proceeds to step S6504. If riot, a cneck is made in
i
step S 6510 to determine if the Act is Change text color. If so, a Font tag is added around the selected text with the foreground color set to the selected color in step S6511

nd proceeds to step S6504. If not, a check is made in step S6512 to determine if Act is Delete, f so, the tag is modified to to hide the selected part of the text in step S6513 nd proceeds to step S6504. If not, a check is made in step S6514 to determine if Act is Undo, f so, the last performed annotation is undone and proceeds to step S6504. If not, a check is nade in step S6516 to determine if Act is UndoAll. If so, all the annotation performed by the iser on this page during this session are removed and proceeds to step S6504.If not a check is nade in step S6518 to determine if the Act is Save, in which case step S6519, CheckExisting explained earlier is executed. In either case the function returns, if no more action needs to be performed by the user. In step S6504, the modified HTML tag page is passed back to the browser, which will render and update the UI in step S6505 and retail to step S650L If Act is lone of the ones specified in the flowchart, the function returns.
Fig. 66 is a flowchart of the procedural steps of S6138 Extract. In step S6601, an instance of the knowledge structure for person - KPTPerson is created. In step S6602, the User operation is set to Act. An example User interface to describe some of the action is shown in Fig. 26. First a check is made to determine the type of Act (Le., if Keep As, Name, Email, Phone, Fax, Notes) was input by the user (steps S6603, S6607~S6610). This action can be performed in various ways, like first selecting the text to be extracted and pressing a predefine button, or dragging and dropping the text to be extracted to the appropriate field or by right clicking on the selected text and specifying it to be the extracted data. If so, the KPTPerson knowledge structure is modified appropriately in step S6604 and the UI gets updated in step S6605 and the process returns to S6602. If the Act is Clear All (step S6611), all the fields are cleared in step S6612 and process proceeds to S66041 If the Act is Save (step S6613), a new action KPTAction is created of type Extract and the KPTPerson is filled as associate object and the process returns.
Fig. 67 is a flowchart of the procedural steps of S6319 Mgmt. In step S6701, Act is set to the user-performed operation. In step S6702, a check is made to determine if Act is NULL. If so, the process returns. If not, a check is made in step S6703 to determine if Act is Show Sessions. If so, all the links are sorted in time order and displayed in step S6704 and the process returns. If not, a check is made in step S6705 to determine if the Act is Show Links. If so, all the links are sorted in the specified order and displayed in step S6706 and the process

returns. If not, a check is made in step S6707 to determine whether Act is Retrieve pages. If Act is to retrieve a page, the process proceeds to step S6708 where RetrieveUI is displayed. If not, a check is made in step S6709 to determine if Act is Retrieve extracted data. If so, Show Retrieve Extracted data UI is displayed. If not, a check is made in step S6711 to determine if Act is Show address book. If so, Show Address book UI is displayed and the process returns. If not a check is made in step S 6713 to determine if Act is Show User Settings. If so, the user settings are displayed to the user in step S6714 and the process returns.
Fig. 68 is a flowchart of the procedural steps of S6022 ExecuteAction. In step S6801, the next Act is got from the ActList. In step S6802, a check is made to determine if Act exists. If not, the process returns. Otherwise, in step S6803 inference is made using the knowledge base to complete the Act A check is made in steps S6804~S6808 to determine if Act is Quick Save or Save or Hold or Automatic Save and if either one of them is true, step S6805, SaveContents as explained later in Fig. 69. is executed and goes to step S6813. Otherwise a check is made in step S6809 to determine if Act is send. If so, SendContents, as explained later in Fig. 69 is executed and goes to step S6813. If not, a check is made in step S6811 to check if Act is Extract. If so, the KPTAction and the corresponding KPTPerson are added to the knowledge base in step S6812 and in step S6813 the knowledge base is updated. A check is made in step S6814 to determine if the completed action needs to be informed to a
i
server for purpose of logging or otherwise. If so, step S6815 is executed wherein the actual information is sent to either a predefined or user specified server about the processing completion and the process returns to step S6801 to fetch the next action from the ActList, till there are no more action left to be processed, at which stage the process returns.
Fig. 69 is a flowchart of the procedural steps of SaveContents in step S6805 of this embodiment. A check is made in step S6901 to determine if it is a SaveLink only operation. If so, process proceeds to step S6905. Otherwise, a check is made to determine if it is a SavePage contents operation in step S6902. If so, Page PLUS is set to true in step S6904. In either case, step S6903, WebFetch is executed, which is explained in detail later in Fig. 70, in step S6903. In step S6905, a check is made to determine if ModifyStatus is saveAsNewAction or not. If so, indices of KPTAction and the associated KPTDocument is determined from Knowledge Base in step S6909 and SaveFileContentjs is executed as

explained in Fig.72, in step S6901. The KPTAction and KPTPerson are added to Knowledge Base in step S6906 and the process returns. If ModifyStatus is not saveAsNewAction, check is made in step S6907 to determine if it is OverWriteExisiting. If not the process returns, otherwise, in step S6911 indices of KPTAction and the associated KPTDocument is determined from Knowledge Base in step S6911 and SaveFileContents is executed as explained in Fig.72, in step S6902. The KPTAction and KPTPerson are updated in the Knowledge Base in step S6908 and the process returns.
Fig. 70 is a flowchart of the procedural steps of WebFetch in step S6903 of this embodiment. In step S7001, HTML document obtained from the browser is opened. In step S7002, next tag is got. In step S7003, a check is made to determine if the end of file has been reached. If so the process returns. If not, a check is made to determine if the tag is for an embedded image, frame etc. in step S7004. If so, step S7005 is executed. If not, a check is made in step S7009 to determine if PagePLUS is true and the Tag typ Fig. 71 is a flowchart of the procedural steps of SendContents in step S6810 of this embodiment. In step S7101, a check is made whether the contents to be sent are SendLink only. If so, a message is created with the given Link in step S7102 anc proceeds to step S7107. If not so, a check is made to determine if the contents to be sent are SendPage (i.e., send the HTML page including the embedded images etc.), in step S7103. If not so, a message for only top HTML is created and proceeds to step S7107. Otherwise, Webfetch is executed as

explained in Fig.70 in step S7105. In step S7107, the created message is sent and the knowledge structures KPT Action and KPTDocument are added to knowledge base in step S7108 and the process returns.
Fig. 72 is a flowchart of the procedural steps S6910, S6912 SaveFileContents of this embodiment. A check is made in step S7201 to determine if the contents to be saved is SaveLink only. If so, the process continues to step S7206. In step S7202, a folder Fl with the name based on the KPTDocument's name, which is a Globally unique identifier (GUID) is created, which ensures that the folder to be created is unique within and across the local system. In step S7203, a file called KPTIndex is created in the folder created in previous step. The actual page contents (i.e., HTML text) are saved to the file created in the previous step. The fully qualified file name (i.e., the folder name and the file name) is stored as the physical URL location of the KPTDocument. In step S7206, FillKPTAction is executed which is explained in detail in Fig. 73 and the other required indices are determined by referring to the knowledge base in step S7207 and the process returns.
Fig. 73 is a flowchart of the procedural steps of S7206, FillKPTAction of this embodiment. In step S7301, the contents of'Keep As' are set to 'Remember As' field of KPTDocument. In step S7302, the contents of 'URL' are set to 'LogicalURL' field of KPTDocument. In step S7303, the contents of'keyword' are set to 'Keyword' field of KPTDocument. In step S7304, the time and date are set to 'WhenDone' field of KPTAction.
In step S7305, a check is made to determine if KPTAction is Save. If so, step S7306 is
j
executed. If not, a check is made in step S7308 to determine if KPTA&ion is Hold. If so, in step S7306, the 'Validity' is set to 'WhenToDo' field of KPTAction ajnd in step S7307, 'Page title' is set to 'Title' of KPTDocument and process returns. If not, step S7309 is executed, in which the ' WhenToDo' field of KPTAction is filled with value 'infinity' and the process returns.
Fig. 74 is a flowchart of the procedural steps of TERMINATE of step S6003 of this embodiment. In step S7401, the UI on display are closed. In step S7402, all the current sessions are ended. In step S7403, Knowledge base is updated. A check is made in step S7404 to determine if browser needs to be ended or terminated. If so, the browser will be terminated in step S7405 and the process ends.

Fig. 75 shows an example of the knowledge structures in the knowledge base. (a), (b), (c) are the knowledge structure definitions for KPTConcept, KPTPerson and KPTDocument respectively.
Fig. 76 shows an example of the knowledge structures in the knowledge base, (a), (b) are the knowledge structure definitions for KPT Action and KBTContent respectively.
Fig. 77 shows an example content of the knowledge base, (a), (b) are the contents of the knowledge base for KPTDocument and KPTAction respectively.
Fig. 78 shows an example content of the knowledge base, (a), (b) are the contents of the knowledge base for KPTPerson and KPTContent respectively.
Fig. 79 is a flowchart of the procedural steps to process the PRINT command. Unlike the Print option associated with the browser, this options lets the user to not only print the currently displayed page, but also allows for reformatting, laying out multiple pages onto a single page etc... When 'Print This' is clicked, for example in Fig. 3, the page contents are obtained from the web browser, in step S7901. In step S7902, a check is made to determine if page layout needs to be done. If so, a check is made in step S7903 to determine, if the parameters required for the layout have already been obtained. If not, the parameters are
i
obtained in step S7904, an example of which is shown in Fig. 82. In either case, the process proceeds to step S7905, wherein the output contents are laid out and process proceeds to step S7906. In S7906, a check is made to determine if the print operation has to be performed immediately. If not, the print command is saved in S7907, and will be picked up later by the daemon process and the process ends. Otherwise, a check is made in step S7908, to determine if a printer for outputting needs to be selected either by the system or the user. If user is to select the printer, step S7909 is executed, wherein the Select Printer UI is displayed, an example of which is shown in Fig. 80. In either case step S7910 is executed, wherein the necessary print data is generated and the actual print data transferred to the printer for printing instepS7911.
Fig. 80 is an example of Select Printer UI, of step S7909. As can be seen from the figure, it is not only possible to select the printer but also specify details of print.
Fig. 81 is an example print output of the web page, which has not been laid out.

Fig. 82 is an example user interface within the web page to obtain the various parameters for PRINT. As can be seen from the figure, it is possible to specify various parameters - number of pages to be printer per physical page, purpose of print etc....
Fig. 83 is an example print output of multiple web pages laid out onto a single page.
Fig. 84 shows another system architecture for this embodiment, when applied to
Information Kiosk, which can be placed in public places like Hotels, Airport, Tourist sites,
City office etc. The information can be maintained on a host server, which can be hosted
separately by an organization or city office etc. It should be noted that traditional methods to
send information by email is not possible, since the information is stored in a private place and
sending only the link or the URL by email is of no use, since the information will not be
accessible from anywhere else. As can be seen from the figure, the input devices can be
Keyboard, Touch panel, Voice etc. and output devices can be either oijie or more of Printer,
i Voice etc., also all the above devices including the hard disk to store the information may be
optionally present.
Fig. 85 shows an example web page displayed on the Information Kiosk along with commands embedded within the web page. As the user navigates through the information on the kiosk, he can choose to collect the information he would like to read later or get it by using the 'Get this' functionality.
Fig. 86 is a flowchart of the procedural steps to process the commands on the Information Kiosk. A check is made in step S8601 to determine if the input command is 'Get this'. If so, step S8602 is executed, wherein a check is made to determine if it is a First time operation. If so, the user settings are displayed in S8603, an example of which is shown in Fig. 87 and the settings obtained from the user in S8604 and the information list initialized in step S8605. In step S8606, the page contents are then obtained from the web browser and added to the List and the UI is refreshed in step S8608 to reflect the changes in the list and the process moves to step S8601. A check is made In step S8608 to determine, if the input is 'Change Settings', in which case step S8609 is executed to display the current settings to the user and any changes to the current settings obtained from the user in step S8610 and saved before returning back to step S8601. A check is made in step S8611 to determine if the input is 'Exit',

in which as check is made in step S8612 to determine if user had collected any information in the list, if not the process ends, otherwise a check is made in step S8613 to determine if a warning message needs to be displayed to the user, if not the process ends. Otherwise the process moves to step S8615, which is also executed if the input is cDone' from step S8614, to output the selected information in the appropriate format as selected by the user. It should be noted that the above process can either be executed on client as explained in Fig. 6 or can be performed on the server as explained in Fig. 37.
Fig. 87 shows an example UI of user settings. As can be seen from the figure, the information of interest can be collected and emailed, along with actual page contents, since the content may reside in a local or private server or get a print out either in AS IS or laying out for example, multiple pages onto a single page etc,.. or get the printed stuff, mailed as printed matter or sent out as a catalog or burn a CD-R or DVD of the information collected etc...
Fig. 88 shows an example UI of user settings for making a CD/DVD for the information selected by the user. As can be seen from the figure, the user can specify for example, the room number to which he would like the CD/DVD to be sent and also details of payment options.
Fig. 89 shows an example UI for making the CD for the information collected by the user. User collects the information he is interested in by pressing the 'Get this!' button and the information currently collected can be seen in the bottom half of the screen and pressing 'Done5 will result making of a CD as per user settings.
Fig. 90 shows another example of web page related to historic information with 'Get this!' button. This can be an E-Commerce site or a shop to make a customized personal catalog matching the interests of the user or customer. User can use the 'Change Media' option to change output form or media at any time. Also, it is possible that the user can choose to get all the information on a same CD or choose multiple CD option or choose a mixture of media, for example, a few pages on CD and rest as printed matter etc.
Fig. 91 shows another example of web page related to product information with 'Get this!' button. Information Kiosk can be set up for e.g., in company headquarters, branch office or showroom, which normally has the information from the intranet, to enable the

customer to collect the product or other non-confidential information regarding the corporation.
Fig. 92 shows an example user settings for printed matter dePvery.
Fig. 93 shows an example information database for this embodiment.
Fig. 94 shows another system architecture for the implementation of a standalone Information Kiosk for this embodiment. As can be seen from the figure, CD-R, Printer and the information database exist within the kiosk 9401. Since this is a standalone kiosk, there is no option to send the contents of the information by email.
Fig. 95 shows another system architecture for the implementation of a networked Information Kiosk for this embodiment. As can be seen from the figure, multiple Information kiosks 9501 are connected on network to multiple Host Servers 9502, which has necessary output generation media like Printer, CD/DVD writer.
Fig. 96 shows example architecture, when multiple clients access the embodiment. Thus if multiple users use the same client then information is stored separately for each login user and is not visible to other users, to protect the privacy of the data. This can be implemented by either storing it in different databases or in separate tables, within a same database.
Fig. 97 is a flowchart of the procedural steps of INITIALIZE for the system shown in Fig. 96. As can be seen from the figure, a check is made in step S9701 to determine if the Instruction is Null. If so, the process ends otherwise, a check is made in step S9702 to determine if the user has already logged in. If so, the instruction is processed in step S9703 and the process returns, otherwise, login screen is shown to the user, an example of which is shown in Fig. 98. A check is made in step S9705 to determine, if the login was successful. If so, the process proceeds to step S9703, otherwise an error message is displayed in step S9706 and the process ends.
Fig. 98 is an example Login UI for the step S9704.
Fig. 99 is a flowchart of the procedural steps of TERMINATE for the system shown in Fig. 96. As can be seen from the figure, in step S9901, the UI on display are closed. In step S9902, all the current sessions are ended. The user is logged otit from the knowledge base in step S9903 and in step S9904, Knowledge base is updated. A check is made in step

S9905 to determine if browser needs to be ended or terminated. If so, the browser will be terminated in step S9906 and the process ends.
Fig. 100 shows another functional block diagram of information processing system embodying the present invention. The information on the Internet 1001 is browsed using multiple browsers 1002A, 1002B simultaneously. Information Management System, KPT System 1003 is for managing information. KPT System interacts and acts as a controlling system as explained in detail in this embodiment to Browser 1002, KPT IMPORT 1007, to import information and KPT EXPORT 10008 to export the information, as explained in detail in this embodiment. Knowledge Base Management 1004 is the management of knowledge accessed/stored from/to the Database 1005. KPT System also handles and processes the information from individual browsers separately as independent sessions.
Fig. 101 is a flowchart of the main procedural steps of KPT System. In step SI0101, initialization processes to connect to the Internet are executed. In step SI0102, main function processing browsing, annotating, saving etc. of this embodiment is performed. In step SI0103, terminate or clean-up processing is executed.
Fig. 102 is a flowchart of INITIALIZE procedure of step S10101. In step SI 0201 a check is made to determine if the browser needs to be instantiated or not. If browser is'not instantiated, it is instantiated in step SI0202. In step SI0203, anew session is created. In step SI0204, the knowledge base is updated. A check is made in step SI0205 to determine if there is any information to be imported. If so, the import data path, KPTPATH is obtained in step S10207 and a request is made to KPTIMPORT to import the information from KPTPATH, in step SI0208. In either case, step SI0206 is executed, wherein the main user interface is displayed and the process ends.
Fig. 103 is another flowchart of the procedural steps of S6319 Mgmt. In step SI0301, Act is set to the user-performed operation. In step SI0302, a check is made to determine if Act is NULL. If so, the process returns. If not, a check is jmade in step SI 0303 to determine if Act is Show Sessions. If so, ShowSession is executed, in which all the links are sorted in time order and displayed in step S10304 and the process returns. If not, a check is made in step SI 03 05 to determine if the Act is Show Links. If so, ShowLinks is executed, in which all the links are sorted in the specified order and displayed in step SI0306 and the

process returns. If not, a check is made in step S10307 to determine whether Act is Retrieve pages. If Act is to retrieve a page, the process proceeds to step SI 03 08 where RetrieveUI is displayed. If not, a check is made in step SI 03 09 to determine if Act is Retrieve extracted data. If so, Show Retrieve Extracted data UI is displayed. If not, a check is made in step SI 0311 to determine if Act is Show address book. If so, Show Address book UI is displayed and the process returns. If not a check is made in step SI 0313 to determine if Act is Show User Settings. If so, the user settings are displayed to the user in step S10314 and the process returns. If not a check is made in step SI 0314 to determine if Act is Import. If so, the import instruction is passed on to KPTIMPORT in step SI0316 and the process returns. If not a check is made in step SI 0317 to determine if Act is Export. If so, the export instruction is passed on to KPTEXPORT in step S10318 and the process returns.
Fig. 104 is a flowchart of the procedural steps of S10304, ShowSession of this embodiment. A check is made in step SI 0401 to determine if all the information needs to be displayed. If not, the process proceeds to step S10402, wherein based on either User settings and/or interaction and/or input and/or System settings, the filter or the constraint on the information to be displayed is obtained. In either case the process proceeds to step SI0403, wherein the relevant KPT Action and the associated KPTDocument are got from Knowledge Base. In step S10404, KPTAction is sorted for each session by time. In step S10405, Session UI is displayed, an example of which is shown in Fig. 115 and the process returns.
Fig. 105 is a flowchart of the procedural steps of S10306, ShowLinks of this embodiment. A check is made in step S10501 to determine if all the information needs to be displayed. If not, the process proceeds to step SI 0502, wherein based on either User settings and/or interaction and/or input and/or System settings, the filter or the constraint on the information to be displayed is obtained. In either case the process proceeds to step S10503, wherein the relevant KPTAction and the associated KPTDocument are got from Knowledge Base. In step SI 0504, a check is made to determine if the Sort Item is equal to Organizations. If so, the information is sorted by Organization, in step SI0505 and proceeds to SI0511, where it is displayed, an example of which is shown in Fig. 114. If not, a check is made in step SI 0506 to determine if the sorting is by Domains. If so, the information is sorted by Domain, in step S10507 and proceeds to step S10511, where it is displayed, an example of which is

shown in Fig. 113. If not, a check is made in step SI 0508 to determine if the sorting is by Keywords. If so, the information is sorted by Keywords, in step SI 0509 and proceeds to step S10511, where it is displayed, an example of which is shown in Fig. 116. If not the information is sorted by requested parameter, in step SI0510 and proceeds to step S10511, where it is displayed, an example of which is shown in Fig. 122, wherein the information is sorted by Table of Contents and the process returns.
Fig. 106 is a flowchart of KPTIMPORT. In step S10601, a check is made to determine if the path KPTPATH, from where the information is to be imported is specified or not. If not, Import UI is displayed by this embodiment to the user. In either case step SI 0603 is executed, wherein information about the import data is obtained and moves to step SI0604 ImportData, details of which are explained in detail in Fig. 107. In step S10605, a check is . made to determine if the results of the import operation are to be notified to the user. If so, step SI0606 is executed to notify the user and the process ends.
Fig. 107 is a flowchart of the procedural steps of ImportData of SI 0604 of this embodiment. In step SI0701 the next information to be imported is obtained. A check is made in step SI 0702 to determine if there is any information to be imported. If not, the process returns. Otherwise, a check is made in step S10703 to determine if the' information is indexed or not. If not, step SI 0704 IndexData is executed to index the information being imported, as explained in Fig. 108. In either case, step S10705 is executed, in which the relevant knowledge structures KPTAction etc. are created anil the indices set in step SI0706. In step SI0707, the actual import operation is performed by passing on the request to KPTSystem along with the created knowledge structures to perform the SaveContent operation as explained in earlier in Fig. 69. The process then returns to SI0702 and continues the import operation till no more information is left to be imported. It is obvious from the figure that it is also possible to first create all the knowledge structures along with indices first in a loop and then pass on save instruction to KPTSystem, to import all the information by a single instruction.
Fig. 108 is a flowchart of the procedural steps of IndexData SI 0704 of this embodiment. In step SI 0801, a check is made to determine if the embodiment can itself index the information. If so, step SI0802 is executed, in which the embodimjent refers to the
knowledge base and decides the indices based on general or specific rples and the process
i

ends. If not, the index information is obtained from the user from step SI 0803 and the process ends.
Fig. 109 is a flowchart of KPTEXPORT of this embodiment. In step SI 0901, Export UI is displayed to the user to obtain the information about what data needs to be exported by this embodiment. In step SI0902, the actual information to be exported is obtained and in step SI0903, the format in which the information needs to be exported is obtained. A check is made in step S10904, to determine if the user will specify the indices for export. If so, Index is set to UserDefined in step SI0906, otherwise Index is set to Default in step SI0905 and the process proceeds to step SI0907, where in ExportData is executed, to export the actual information as explained in Fig. 110. In step SI0908, a check is made to determine if the results of the export operation are to be notified to thq user. If so, step SI 0909 is executed to notify the user and the process ends.
Fig. 110 is a flowchart of the procedural steps of ExportData, S10907 of this embodiment. In step SI 1001, the next information to be exported is obtained. In step SI 1002, a check is made to determine if there is any more information left to be exported. If not the process ends. Otherwise, step SI 1003 is executed, wherein a check is made to determine if the Index is UserDefined. If not step SI 1004 is executed and the indices are obtained from the user and proceeds to step S11006. Otherwise, step S11005 is executed to obtain the indices from the KPTAction knowledge structure or the knowledge base. In step S11006, the actual information is exported in the specified format and the process loops back to step SI 1001, to fetch the next information to be exported, till there are no more information left to be exported, at which the process ends. It is obvious from the figure that it is also possible to first create all the knowledge structures along with indices first in a loop and then export all the information in a single operation..
Fig. 111 is an example of Import UI. As can be seen fr6m the figure, the information type to be imported is obtained from the user. For example, this could include, bookmarks or favorite URL(s) stored within a web browser, a marked up file like SGML, XML, a set of files or folders or other information, details of which will be input by the user.
Fig. 112 is an example user interface of browsing files to import. In the example ImportUI of Fig. 111, the user can either directly input the name(s) of files/folders to

be input or could select the Browse button to specify the name of the files/folders from the list of files in the underlying operating system.
Fig. 113 is an example of Show all Links sorted by Domain. The example, shows the information with the embodiment automatically sorted by Domains. For example, the Tokyo University information saved from www.u-tokyo.ac.jp is automatically shown below Japan, Universities based on the knowledge base rules.
Fig. 114 is an example of Show all Links sorted by Organizations. The example, shows the information with the embodiment automatically sbrted by Organization name. The information was imported from the web browser bookmarks.
Fig. 115 is an example of Show Sessions. This example, shows the information with the embodiment in time sorted order, based on for e.g., when the action was performed, the validity of information or any other time based parameter. As can be seen from the figure, the right hand side displays an example XML file, imported into the system.
Fig. 116 is an example of Show all Links sorted by Keywords. As can be seen from the figure, the imported files/folders are sorted based on the keywords and the imported information is in a separate node below 'My Keepoint' and the web information being saved by the embodiment is stored in a separate node called 'Web Information'. When multiple keywords have been assigned to information, unlike current filing systems, the embodiment allows the same document to occur below multiple keywords.
Fig. 117 is another example of Show Sessions, in which the imported conference sessions are shown. As can be seen, the 'MW 2002' conference proceedings are shown below the actual date, when the conference was held rather than when the information was saved by the embodiment. The figure also shows the Extract Data UI to extract address book related data extraction, as was explained earlier in detail.
Fig. 118 is another example of Show all Links sorted by Keywords, for imported conference information. As can be seen, the same proceedings information 'MW 2002 Proceedings', can be viewed from multiple nodes - 'Boston', 'Proceedings' etc..., since the information contains multiple keywords. The figure also shows the Save UI for saving the information, assigning/ modifying keywords, page titles etc...

Fig. 119 is a warning dialog to prompt the user to insert the CD containing the full proceedings of the conference. As was explained earlier, it is possible that only the indices md a subset of information are copied by the embodiment and the full information resides on in external device or media like CD-ROM or Server, then the user is prompted to insert for s.g., the CD containing the full information, if such a media cannot be found by the embodiment. The embodiment remembers the path but it is also possible to specify a new path or a new server name, since it is likely that such things change over time.
Fig. 120 is another example of Show all Links sorted by Keywords, for imported conference information, along with a simplified Save UI.
Fig. 121 is another example of Show all Links sorted by Organizations, for imported conference information. The example Annotate UI is also shown in this figure, to enable direct annotation of adding notes, adding foot notes, highlighting text, changing text color, deleting information etc.
Fig. 122 is another example of Show all Links, sorted by Table of Contents. As can be seen from the figure, the contents of the proceedings are shown as chapter, sub chapters. Also, ince different types of information groups are shown in the Get All Links or the Sessions view, all the indices between the different types of information are not the same. Hence for example, in the figure, the first type of information is a Conference Proceedings and as can be seen from the figure, the contents of that node are sorted by Table of Content view. However, the web information saved by the user neither have any indices to for table of content type of sort nor does it make any sense to show the information gathered over a period of time. Hence the sort for the web information is not changed and shown either in the default sorted order or in the previous sorted order.
Fig. 123 is another example of Show Sessions, in which the conference information and other web information saved are shown in separate nodes. It is also possible that the information can be predefined and loaded into the KB of the KPT System. For example, as can be seen from the figure, the conference proceedings are preloaded into the KPT System. Thus when the user views the information using the KPT System, he can view the conference proceedings in a separate node and the web information he saves in a separate

node. Also, it may be possible that only the top-level pages and the indices might be stored with KPT system and the actual detail information or pages may be on a CD-ROM.
Fig. 124 is another example of Show iill Links, sorted by Keywords, in which the conference information and other web information saved are shown in separate nodes.
Fig. 125 is another example of Show all Links, first sorted by Organization and then by Authors below each node. As explained earlier, since different types of information exists with the embodiment, the conference proceedings, in this figure are sorted by Organization, followed by Authors for each of the Organization. However, for the web information stored by the user, the information is categorized only by Organizations.
Fig. 126 is another example of Show all Links, sorted by Authors and Organization. In this example, the Author and Organization are at the same level i.e., the information is sorted using both the indices at the same level, as it is possible that we may either remember vaguely the name of the person or the name of the organization. In previous figure, the user had to know the organization name, to locate the information even if he knew the author name. It is possible to have several other obvious combinations like the information is sorted by Author.
Fig. 127 is an example user interface of AutoFetch List. As can be seen from the figure, the list of information to be fetched are shown. It is possible to add a new instruction using the Add, edit the instruction using Edit or delete the instruction using Remove.
Fig. 128 is an example user interface of specifying a new auto fetch instruction. As can be seen from the figure, the name, the URL to fetch the information from, where to save, report to generate, time to get etc. can be specified.
Fig. 129 is an example user interface for specifying the! output report to be generated. As can be seen, it is possible to present the accumulated data in reports for e.g., Graphs or charts.
Fig. 130 is an example user interface to specify the information sources to look for.

Fig. 131 is an example of Show Sessions of the auto fetch results. As can be seen from the figure the stock price information of an company is collected every day and presented in raw form.
Fig. 132 is another example of Show Sessions of the auto fetch results in a graphical format. The stock price collected is displayed in a graphical form for easy viewing.
Fig 133 is another example of Show Sessions of the auto fetch results in a graphical format.
Fig 134, 135 and 136 are examples of Show Sessions of the several imported information. As can be seen in Fig. 134, the speeches of Abraham Lincoln have been collected and shown in the time sorted order of when they were delivered, rather than when the information was collected by the user. Fig. 135 shows an example of all the product information collected for a particular company. Fig. 136 shows an example of some of the technical articles collected on Java programming language.
Fig. 137 is another example of Get all Links for imported association members. As can be seen from the figure, the members of the associate — individuals and organizations have been imported and shown in Get All Links view. The right side displays the details of member information.
Fig. 138 is an example user interface of Export. As can be seen from the figure, it is possible to select multiple information to be exported.
Fig. 139 is an example user interface to specify the information to be exported. As can be seen from the figure, it is possible to specify a complete node or only specific nodes.
Fig. 140 is an example user interface to specify some other parameters for export. As can be seen from the figure, it is possible to specify how the information needs to be exported for example, in Keepoint format with indices to other embodiment of similar nature, as data along with program, which can be viewed using a standard web browser i.e., the data is in HTML format and the program to sort the views in some scripting language like JavaScript or VBScript etc... It is also possible to specify if the indices needs to be exported along with the information being exported.

Fig. 141 is an example user interface to specify the format of the information to be exported. As can be seen from the figure, the Export Data format c^n be specified for e.g., SQL Database format, Text File, XML file, CSV format etc...
Fig. 142 is an example of exported data in a browser independent format. As can be seen from the figure, the exported data can not only be viewed from a standard web browser but it is possible to sort the information using various indices. This is done for e.g., by the JavaScript program embedded in the information.
Fig. 143 is another example of exported data, in which the type by which the information is to be sorted is obtained using a pull down menu type of user interface.
Fig. 144 is another example user interface to specify the information to be exported. As can be seen from the figure, the data extracted using this embodiment has been selected.
Fig. 145 is an example output of the exported address book data.
Fig. 146 is an example output of the exported address book data in a comma separated CSV format.
Fig, 147 is an example output of the exported address book data in Text format.
Fig. 148 is an example output of the exported address book data in HTML format.
Fig. 149 is example information as seen in a browser of the exported address book data.
Fig. 150 is an example of viewing the published information from this embodiment. It is also possible to export information from this embodiment to allow publishing the information, for e.g., on a web server or in form of printed information.
Fig. 151 is an alternative flowchart of the procedural steps of SI 0304 ShowLinks or S10306 ShowSession of this embodiment. In step S15101, the parameter in which the information is to be sorted, SORTB Y is either obtained from the user or from system settings. In step S15102, a check is made to determine if all the information needs to be displayed. If not, the nodes to be displayed DISPNODES are either obtained from the user or from settings in step SI5103. In either case, step SI5104 CreateTree is executed, which is explained in detail later in Fig. 152. The actual tree view is displayed in step S15105

ShowTreeView, which is also explained in detail later in Fig. 154. When the user closes the view, the process returns.
Fig. 152 is a flowchart of the procedural steps of CreateTree. In step SI 5201, the NodeList is set to NULL. In step S15202, the next node Nl is obtained from the list of , nodes to be displayed, DISPNODES. A check is made in step SI 5203 to determine, if Nl exists. If all the nodes in the DISPNODES list has been exhausted, then step SI 5204, SortTreeView is executed to sort the information based on SortBy, as explained in Fig. 153 and the process returns. Otherwise, step SI5205 is executed to obtain all the relevant list LI for node Nl from the knowledge base. In step SI 5206, the next node information Kl is obtained from list LI. A check is made in step SI 5207 to determine if]Kl exists. If node Kl does not exist, the control goes back to step SI5202 to fetch the next node from DISPNODES. Otherwise, step SI 5208 is executed to determine, if the information Kl is unnecessary information. If so, the process loops back to S15206, to obtain the next information from L1. If not, step SI 5209 is executed, where in a check is made to determine if Kl already exists in nodelist If so, the process loops back to S15206, to obtain the next information from L1. Otherwise, step SI 5210 is executed, in which the Kl is added to the appropriate place in NodeList and process loops back to SI5206, to obtain the next information from LI, till there is no more information left, at which stage the process returns.
Fig. 153 is a flowchart of the procedural steps of SortTreeView. In step S15301, the a check is made to determine if SortBy is SESSION, in which case step S15302 is executed to sort all the action by time and process proceeds to step S15312. If not, a check is made in step SI 5303 to determine if SortBy is Organizations, in which case step SI 5304 is executed to sort all the actions by Organizations. If not, a check is made in step S15305 to determine if the SortBy is Domains, in which case step SI 5306 is executed to sort all the actions by Domain. If not, a check is made in step SI 5307 to determine if the SortBy is Keywords, in which case step SI 5308 is executed to sort all the actions by Keywords. If not a check is made in step SI 5309 to determine if there is a need to sort by the requested parameter. If so, step SI 5301 is executed and all the actions are sorted by the requested parameter, otherwise step SI5311 is executed to sort by the default parameter, which could either be from user settings or from the previous parameter by which the sort was performed. In all of the

above cases, step S15312 ShowTreeView is executed, which is explained in detail in next Fig. 154. and the process returns.
Fig. 154 is a flowchart of the procedural steps of ShowTreeView SI 5312 of this embodiment. First in step SI 5401, a check is made to determine if Type is Keyword. If so, No keywords is added to the NodeList in step S15402. In step S15403, the list of nodes in the NodeList is displayed. In step SI5404 the process waits for user operation or Action Act and in step SI 5405, a check is made to determine if the Act is End, in which case the process returns. If not, a check is made in step SI 5406 to determine if a Leaf was selected. If so ProcessLeafAction(Act, Node, Type) is executed in step SI 5407. If not, ProcessNodeAction(Act, Node, Type) is executed in step SI5408 and the process returns to step SI5404.
Fig. 155 is a flowchart of the procedural steps of ProcessLea£Action(Act, Node, Type) of step S15407 of this embodiment. A check is made in step S15501, if the Act is Open. If so, all the child nodes and all the actions KPTAction and associated KPTDocument are fetched in step SI5502, from the knowledge base for the selected node and added to the NodeList at appropriate places in step SI 5503 and continues to step SI 5509. If not, a check is made in step S15504, if the Act is Close. If so, all the child nodes below the selected node are closed or hidden in step SI 5505 and continues to step SI 5509. If not, a check is made to determine if the Act is Delete. If so, a confirmation is sought from the user, if required, in step S15507 and if delete is not to be performed, it continues to step S15509, else all the KPTAction and associated KPTDocument for all the child nodes below the selected node is deleted from the knowledge base in step SI5508 and continues to step SI5509. In step S15509, the knowledge base is updated based on the type of action performed and in step S15 510 the user interface is updated to reflect the updates made in the knowledge base. If in step SI5506, the action is not Delete, the process returns.
Fig. 156 is a flowchart of the procedural steps of ProcessNodeAction(Act, Node, Type) of step SI 5408 of this embodiment. A check is made in step SI 5601 to determine if the Act is Display i.e., to display the contents of the stored page, if contents are stored, otherwise, the original page needs to be displayed. If so, all the KPTAction and associated KPTDocument are fetched from the knowledge base for the selected node and added to the

NodeList at appropriate place in step SI5602 and a check is made in sjtep SI5616 to determine if the information is not with the system of this embodiment and that no connection could be made to the source of the information, then the user is notified in step SI 5617 and proceeds to step S15614. Otherwise also step S15614 is executed, wherein the knowledge base is updated. If not, a check is made in step S15603 to determine if the Act is Source i.e., to display the contents of the original page. If so, the KPTAction and associated KPTDocument are fetched from the knowledge base for the selected node in step SI 5604 and fetches the contents of the page from the original location or URL in step SI 5605 and continues to step SI 5614. If not, a check is made to determine if the Act is Delete in step SI 5606. If so, a confirmation is sought from the user, if required, in step" SI 5607 and if delete is not to be performed, it continues to step SI5614, else in step SI5608, the associated KPTAction and KPTDocument are deleted from the knowledge base and continues to step SI 5614. If not, a check is made in step S15609 to determine if the Act is Delete from this group. If so, a confirmation is sought from the user, if required, in step SI 5610 and if delete is not to be performed, it continues to step SI 5614, else in step SI 5611, the associated attributes or properties of KPTAction and KPTDocument are modified in the knowledge base and continues to step SI 5614. If not, a check is made in step S15612 to determine if the Act is Show Property. If so, the associated properties or attributes of the KPTAction and KPTDocument for the associated node are fetched from the knowledge base in step SI5613 and continues to step SI5614. In step SI5614, the knowledge base is updated based on the type of action performed and in step SI5615 the user interface is updated to reflect the updates made in the knowledge base. If in step S15612, the action is not Show Property, the process returns.
It is possible that some data is in KB of KPT System and the rest of the data is in an external media like CD-ROM or on network like file server etc... For e.g., the index pages and top-level pages, which are used often, can be kept with KPT System and rest of the pages not used often can be in CD ROM. For example, in the case of the proceedings of a conference, the index and top level pages can be stored with KPT System but the actual details of papers presented, author biography etc. can be stored in a CD-RON! as they are less frequently used. In SI 5516, a check is made to determine if some data exists in outside, and if the data can be accessed, for e.g., the CD may not be inserted or may not be connected to

network, the user will be prompted as shown in Fig. 119 to insert the CD. Of course, using the KPT System, the user can choose and save the most relevant pages to the KPT System thus eliminating the need to always insert the CD for pages he or she uses frequently.
Fig. 157 shows an example UI for Importing Email or Documents, when import
other is selected in Fig 111. As can be seen from the figure, either all the Emails can be
imported or only a selected set can be imported using the "specify details", an example of
which is shown in Fig 158. Files or documents can also be imported in a similar fashion, by
either specifying the document to import or using a filter to fetch the matching files e.g., by
name or by type i.e., all Word Processed documents, and/or by date. Also, the emails or files
or documents can be imported either in ASIS or can be converted to HTML during the import
process. '
Fig. 158 shows an example UI for specifying the mails that needs to be imported into the current embodiment. As can be seen from the figure, the user can either import all the mails from a specified folder or can specify a complex condition using various parameters. For e.g., From, To, Subject and date.
Fig. 159 shows another flowchart of SaveContent to enable importing of non-HTML documents into the current embodiment. The main flow is similar to Fig. 69 of this embodiment and a check is made in step S15913 to see if the imported document content is a HTML page, in which case WebFetch0 is called, which was explained in detail in Fig. 70. Otherwise, FetchOtherO is called in step S15914, details of which are explained in Fig. 160. Other steps of this Figure are similar to Fig. 69 of this embodiment.
Fig. 160 shows the flowchart of FetchOther of this embodiment. The list of contents to be saved are first obtained in step SI6001 and instep S160b2, the next content to be saved C is obtained. A check is made in step SI6003 to determine if d is NULL, in which case the process returns. Otherwise, a check is made in step SI6004 to determine if C is a file. If not, the contents are obtained from the parent or owner application in step S16011 and process proceeds to step S16008. Otherwise, a check is made in step S16005 to determine if C is a local file, in which case step SI6007 is executed to copy the file contents, otherwise step SI6006 is executed and the contents are downloaded. In either case SI6008 is executed to check if the contents needs to be converted to HTML. If so, the contents are converted to

HTML in step SI6009. In either case step SI6010 is executed, wherein the contents are added to the knowledge base and the process returns to step SI6002 to obtain the next content C from the list of contents, till no more contents are left in the list. If fetch type is a file, then the contents needs to be only downloaded or copied otherwise the contents or data to be imported needs to be first obtained from the underlying application. For example, during import of email, the email data needs to be obtained from the mailing application. It may so happen that the email application may store the data in normal file, in which case the embodiment can fetch it as normal file. Also several ways exist to obtain the information from the parent or underlying application, for example, Dynamic Data Exchange (DDE), OLE Automation, COM, COBRA etc.
Fig. 161 is an example of Get all Links sorted by Sender for the imported email information. As can be seen from the figure, the imported emails are shown sorted by Sender.
Fig. 162 is another example of Get all Links sorted by Sender for the imported email information.
Fig. 163 is an example of Show Sessions for the imported email information. The mail list is shown in tabular form.
Fig. 164 is an example of Get all Links sorted by Name for the imported documents. The various documents are shown below the Documents node, and the contents of the document are shown on the right hand side.
Fig. 165 is another example of Get all Links sorted by Name for the imported documents, which is imported in AS IS format When user views the application in the browser, the relevant application, for e.g., word processor is launched within the browser as shown in the figure.
Fig. 166 is a flowchart of the procedural steps of another implementation of importData for this embodiment. In step SI 6601, the type of the information being imported is obtained using GetType, which is explained in detail in Fig. 167 and in step S16602, the next information to be imported is obtained. A check is made in step SI 6603 to determine if the Import information exists. If not the process ends otherwise a check is made in step SI 6604 to determine if the information is indexed. If not, step SI 6605 IndexData is executed, which was explained earlier in Fig. 108. In either case step SI6606 is executed to create the KPTAction

knowledge structure and in step SI6607, Setlndex is executed, which lis explained in detail in Fig. 168. In step SI 6608, the necessary knowledge structures are created and a request is made to KPTSystem of this embodiment to execute SaveContents for the created contents and loops back to step SI6602 to fetch the next information to be imported, till there are no more information left to be imported.
Fig. 167 is a flowchart of the procedural steps of GetType. In step SI 6701, the attributes of the information being imported is obtained and set to ATTR. In step SI 6702, a check is made to determine if ATTR is BOUND_DATA. If so, BOUND JDATA is returned. If not, a check is made in step SI6703 to determine if the system can decide based on the ATTR information. If so, a check is made in step SI 6704 to determine if the information needs to be managed together with other information. If so, the process returns BOUNDED AT A, otherwise INDEPENDENT-D ATA is returned. If the system cannot decide based on the ATTR information, the necessary information is obtained from the user in step SI6705 and if the information is to be managed with other information, BOUND_D^TA is returned, otherwise INDEPENDENTJDATA is returned.
Fig. 168 is a flowchart of the procedural steps of Setlndex. A check is made in step SI 6801 to determine if Type is BOUNDJDATA. If so, step SI 6802 is executed in which the indices from the general knowledge base, otherwise the indices are obtained from the individual knowledge base and the process ends.
The present invention described above may be applied to a system constituted of a plurality of computers, or a specific computer within a system. The object of the present invention can also be achieved by supplying a storage medium storing program codes of software for implementing the function of the above embodiment to a system or an apparatus, and reading out and executing the program codes stored in the storage medium by a computer (or a CPU or MPU) of the system or apparatus. In this case, the program codes read out from the storage medium implement the function of the present invention, and the storage medium storing these program codes constitutes the invention. Also, besides the function of the above embodiment is implemented by executing the readout program codes by the computer, the present invention includes a case where an OS (Operating System) or the like running on the

computer performs a part or the whole of actual processing in accordance with designations by the program codes and thereby implements the function of the above embodiment.
Furthermore, the present invention also includes a case where, after the program codes read out from the storage medium are written in a memory of a function extension board inserted into the computer or of a function extension unit connected to the computer, a CPU or the like of the function extension board or function extension unit performs a part or the whole of actual processing in accordance with designations by the program codes and thereby implements the function of the above embodiment.
Although the present invention has been described in its preferred form with a certain degree of particularity, many apparently widely different embodiments of the invention can be made without departing from the spirit and scope thereof. It is to be understood that the invention is not limited to the specific embodiments thereof except as defined in the appended claims.




CLAIMS
1. An information processing method, wherein a web page on the internet is accessed and displayed by an internet browser, characterized by the steps of:
obtaining from the displayed web page a processing instruction for processing the displayed web page; and
transferring the obtained processing instruction to a processing program which is able to execute the processing instruction.
2. The method according to Claim 1, wherein in said obtaining step, the processing instruction is obtained in response to an operation of a button or a mark provided on the page in correspondence with the processing instruction.
3. The method according to Claim 1, further comprising the processing step of executing the processing instruction by the processing program.
4. The method according to Claim 3, wherein said processing step including the steps of:
assigning to the web page a predetermined index containing an identifier of the page; and
storing the assigned index to a database.
5. The method according to Claim 3, wherein in said processing step, the

displayed web page is saved.
6. The method according to Claim 5, wherein in said processing step, a
search is performed in. the saved web page.
7. The method according to Claim 3, wherein in said processing step, the
displayed web page is sent.
8. The method according to Claim 3, wherein in said processing step, an
annotation to the displayed web page is performed.
i >
9. The method according to Claim 3, wherein in said processing step, data is
extracted from the displayed web page.
10. The method according to Claim 3, wherein in said processing step, the displayed web page is printed.
11. The method according to Claim 3, wherein in said processing step, the execution of the processing instruction is inhibited if the page is not within a predetermined site.
12. The method according to Claim 3, wherein in said processing step, the execution of the processing instruction is inhibited if a number or quantity of saved data exceeds a limitation.

13. The method according to Claim 3, wherein the processing program is embedded in the page.
14. The method according to Claim 13, wherein in said processing step, the result of executing the embedded program is transferred to a process independent of the embedded program.
15. The method according to Claim 1, further comprising the steps of:
determining whether or not a processing program for executing the processing
instruction exists; and
requesting to obtain the processing program if the program is not exist.
16. The method according to Claim 1, wherein the processing program is in a
server.
17. The method according to Claim 16, further comprising the steps of:
informing a type of the processing instruction to the server; and
the type of the processing instruction is registered in correspondence with an address of the page in the server
18. The method according to Claim 1, wherein in said obtaining step, one of a
first and second save instructions is obtained in response to an operation of one of a first and
second save buttons, respectively, and in said transfer step, the first save instruction is transferred

to a process for saving the page and the second save instruction is transferred to a process for saving a link to the page.
19. The method according to Claim 1, wherein said transferring step is
performed by a program embedded in the page.
20. The method according to Claim 1, further comprising the steps of:
obtaining data displayed by a browser;
importing data without displaying the data; and
storing and managing the obtained data and the imported data.
21. The method according to Claim 20, wherein the imported data is a book mark or a URL.
22. The method according to Claim 20, wherein the imported data is a text described in a markup language.
23. The method according to Claim 20, wherein the imported data is a folder including files of a plurality of layers.
24. The method according to Claim 20, further comprising the indexing step for assigning a index to the imported data.

25. The method according to Claim 24, wherein in said indexing step, an
index of data is maintained if the index has already been assigned to the data.
26. The method according to Claim 24, further comprising the steps of:
sorting indices assigned in said assigning step; and
displaying the sorted indices on a display.
27. The method according to Claim 20, wherein in said importing step, the
data is imported from a specified URL at a designated time.
28. The method according to Claim 20, further comprising the steps of:
assigning a index to the imported data;
sorting assigned indices in each folder; and displaying the sorted indices on a display.
29. The method according to Claim 20, wherein in said importing step, the
data having a predetermined index is imported from a whole of a network.
30. The method according to Claim 1, further comprising the steps of:
obtaining data displayed by the browser;
storing and managing the obtained data; and exporting the managed data.

31. The method according to Claim 30, further comprising the indexing step for assigning a index to the data to be exported.
32. The method according to Claim 30, further comprising the generating step for generating a program for sorting the assigned indices and displaying the sorted indices in another browser and in said exporting step5 the program is exported with the data to which the index is assigned.
33. The method according to Claim 30, wherein in said exporting step, the data is exported to a specified database.
34. The method according to Claim 33, wherein in said exporting step, the data is processed so that the data can be stored in the specified database.
35. The method according to Claim 30, wherein in said exporting step, the data is modified so that the data can be easily opened to the public.
36. The method according to Claim 30, further comprising the steps of:
importing data without displaying the data; arid
managing the imported data so that the data can be exported.
37. An information processing apparatus, in which a web page on the internet
is accessed and displayed by an internet browser, characterized by:

obtaining means for obtaining from the displayed web page a processing instruction for processing the displayed web page; and
transfer means for transferring the obtained processing instruction to a processing program which is able to execute the processing instruction.
38. A computer-executable program for controlling a computer to perform information processing, said program comprising codes for causing the computer to perform:
obtaining from the displayed web page a processing instruction for processing the
displayed web page; and
transferring the obtained processing instruction to a processing program which is
able to execute the processing instruction.
39. An information processing method, wherein a web page on the internet is
accessed and displayed by an internet browser, substantially as herein described with
reference to the foregoing examples.
40. An information processing apparatus, in which a web page on the internet is
accessed and displayed by an internet browser, substantially as herein described with
reference to the foregoing examples.
41. A computer-executable program for controlling a computer to perform
information processing, substantially as herein described with reference to the foregoing
examples and accompanying drawings.
Dated this 11th day of October 2004.

Documents:

2308-CHENP-2004 CORRESPONDENCE OTHERS.pdf

2308-CHENP-2004 CORRESPONDENCE PO.pdf

2308-CHENP-2004 FORM-1.pdf

2308-CHENP-2004 FORM-18.pdf

2308-CHENP-2004 FORM-2.pdf

2308-CHENP-2004 FORM-3.pdf

2308-CHENP-2004 PCT.pdf

2308-CHENP-2004 PETITIONS.pdf

2308-CHENP-2004 POWER OF ATTORNEY.pdf

2308-CHENP-2004 OTHER PATENT DOCUMENT 04-12-2009.pdf

2308-chenp-2004-abstract.pdf

2308-chenp-2004-claims.pdf

2308-chenp-2004-correspondnece-others.pdf

2308-chenp-2004-correspondnece-po.pdf

2308-chenp-2004-description(complete).pdf

2308-chenp-2004-drawings.pdf

2308-chenp-2004-form 1.pdf

2308-chenp-2004-form 26.pdf

2308-chenp-2004-form 3.pdf

2308-chenp-2004-form 5.pdf


Patent Number 247437
Indian Patent Application Number 2308/CHENP/2004
PG Journal Number 15/2011
Publication Date 15-Apr-2011
Grant Date 07-Apr-2011
Date of Filing 12-Oct-2004
Name of Patentee SAORA KABUSHIKI KAISHA
Applicant Address HIGASHITERO 1-30-40-333, TSURUMI KU, YOKOHAMA-SHI , JAPAN 230-0077
Inventors:
# Inventor's Name Inventor's Address
1 SUDA, ARUNA, ROHRA HIGASHITERO 1-30-40-333, TSURUMI KU, YOKOHAMA-SHI , JAPAN 230-0077
PCT International Classification Number G06F17/30
PCT International Application Number PCT/IB03/00894
PCT International Filing date 2003-03-11
PCT Conventions:
# PCT Application Number Date of Convention Priority Country
1 2002-14103 2002-05-16 Japan
2 2002-111801 2002-04-15 Japan
3 2002-067868 2002-03-13 Japan
4 2002-198009 2002-07-05 Japan