Finding parsed data in output

[luukvancampen]

Hello all,

As part of my computer science master thesis I am examining the output of IPED. I was, however, not able to discover the results of parsing files in the output of IPED. I did of course find the CSV file listing all the discovered files, but I have not been, for example, able to find a file containing data about emails, such as their sender and receiver and topic. I do expect this data to be somewhere, since the IPED user interface also seems to show it in the screenshots on the Github page. I have executed the tool using Docker on an image that contains a wealth of different files.
TLDR: where can I find structured output data from IPED?

Thanks very much in advance!

If I need to provide some more information, please let me know.

Kind regards,

Luuk van Campen
The Netherlands

[wladimirleite]

Hi @luukvancampen!

Not sure if I fully understand your question, but in the "regular usage" of IPED, all parsed/extracted/generated information is avaliable through the analysis GUI.
The CSV generated during the case processing contains some basic fields only.

It is possible to select some items (or all of them), and export their properties to a CSV file. In that case, all visible columns will be included in the CSV. And you can select the visible columns to include the properties you are interested in. Text extracted from the items will not be exported using this procedure, only properties.

Another option would be using the WebAPI.

[luukvancampen]

Hi!

Thank you for the quick reply! So just to confirm, if I understand correctly, there is no way to get those properties without using either the GUI or the WebAPI? I'm interested in automating running the tool and processing the output, so that's the reason for the question.

[wladimirleite]

Not sure, but it may be possible to do what you want using the WebAPI.
Another option would be writing a custom script task, so you can export exactly what you need in the proper format, but that is not trivial and would require some coding.

[lfcnassif]

Hi @luukvancampen. IPED stores parsed data in a structured way into a Lucene index. To access its contents without dealing with low level Lucene API, you can use the Web API (https://github.com/sepinf-inc/IPED/wiki/Web-API) suggested by @wladimirleite or use a python-java bridge like #875 (this uses old IPED v3 API, I'll update the examples to IPED v4 API when I return back from vacation in mid January).

[luukvancampen]

Hello!
Thanks all for the quick and useful answers! With the information that the data is stored in a Lucene index I managed to solve my problem and write Java code that exports the data to JSON.

Thank you very much!