Related Information
Technical Information
|
IntuScan™ Named Entity Recognizer (NER)
IntuView has developed a propriety algorithm for Contextual Information Tagged Entity Identification.
This technology takes advantage of IntuView's own language NLP (Natural Language Processing) algorithms to
extract entities mentioned in the document and to determine whether they are:
people, places, institutions or organizations, URLs and dates (Gregorian, Jewish, Farsi, Hijri).
IntuView's algorithms do not only identify and link together entities which appear in the same form in the same document but identifies the same entity when it crops up in different documents in the same batch, and even in variant forms.
For example, a person who is mentioned in a set of documents in different forms as follows:
Will all create the entity: Ahmad Yousuf Muhammad, described in the texts as Mujahid, Sheikh, Brother (in reference to the Islamic Army), Leader of the Islamic Army. His ideological orientation is jihadi-salafi and he is associated with the Islamic Army. He appears to have been alive on 1.2.2009 but deceased before 1.4.2009.
Similarly, the following names extracted from the document:
All will be presented in the report as: Abdallah Yousef 'Azzam, AKA Abu Yousef 'Abdallah who is titled in the document as Sheikh, Brother, Sheikh Mujahid, Martyr Sheikh and is described as "the founder of the Jihadi movement".
The information gleaned from different occurrences of the person – even under variant names – is aggregated into an incrementally comprehensive picture of the person – his titles, his sect, his political orientation, etc. This "ad-hoc" person is added to the IntuScan knowledge base for further reference in other texts.
Automatic Transliteration
Automatic transliteration of named entities in Arabic and other relevant languages, is another challenging task. The fact that Arabic names are written without diacritics, makes this task even more difficult. We use sophisticated rule based algorithm for dealing with different complicated cases. Beside the internal IntuView's transliteration system, IntuScan™ Named Entity Recognizer supports the following standard transliteration systems:
IC, BGN, SATTS.
people, places, institutions or organizations, URLs and dates (Gregorian, Jewish, Farsi, Hijri).
IntuView's algorithms do not only identify and link together entities which appear in the same form in the same document but identifies the same entity when it crops up in different documents in the same batch, and even in variant forms.
For example, a person who is mentioned in a set of documents in different forms as follows:
- Mujahid Sheikh Ahmad Yousuf Muhammad
- Sheikh Abu Yousuf , Leader of the Islamic Army
- Brother Mujahid Abu Yousuf Ahmad, MAY ALLAH PRESERVE HIM
(document date 1.2.2009) - Our Leader, Sheikh Ahmad Yousuf Muhammad, MAY ALLAH HAVE COMPASSION ON HIM
(document date 1.4.2009, document identified as belonging to the Islamic Army)
Will all create the entity: Ahmad Yousuf Muhammad, described in the texts as Mujahid, Sheikh, Brother (in reference to the Islamic Army), Leader of the Islamic Army. His ideological orientation is jihadi-salafi and he is associated with the Islamic Army. He appears to have been alive on 1.2.2009 but deceased before 1.4.2009.
Similarly, the following names extracted from the document:
- Sheikh 'Abdallah 'Azzam
- Brother Sheikh 'Abdallah 'Azzam, founder of the Jihadi Movement
- Sheikh Martyr 'Abdallah Yousuf 'Azzam, MAY ALLAH HAVE COMPASSION ON HIM
- Martyr Sheikh Abu Yousuf 'Abdallah MAY ALLAH HAVE COMPASSION ON HIM
All will be presented in the report as: Abdallah Yousef 'Azzam, AKA Abu Yousef 'Abdallah who is titled in the document as Sheikh, Brother, Sheikh Mujahid, Martyr Sheikh and is described as "the founder of the Jihadi movement".
The information gleaned from different occurrences of the person – even under variant names – is aggregated into an incrementally comprehensive picture of the person – his titles, his sect, his political orientation, etc. This "ad-hoc" person is added to the IntuScan knowledge base for further reference in other texts.
Automatic Transliteration
Automatic transliteration of named entities in Arabic and other relevant languages, is another challenging task. The fact that Arabic names are written without diacritics, makes this task even more difficult. We use sophisticated rule based algorithm for dealing with different complicated cases. Beside the internal IntuView's transliteration system, IntuScan™ Named Entity Recognizer supports the following standard transliteration systems:
IC, BGN, SATTS.