Recruitment Project for 2nd year.
System that generates summarised text for the text available in literary form of study materials like pdf, etc along with natural language captions for any image.
The image captioning model is trained on Flickr8k Dataset
The model has been trained for 20 epoches on 6000 training samples of Flickr8k Dataset.
- Python 3.6
- sumy
- pdfplumber
- fitz
- tensorflow
- pillow
- matplotlib
- h5py
- keras
- numpy
- pydot
- nltk
- progressbar2
- pytesseract
- PyPDF2
- io
These requirements can be easily installed by:
pip install -r requirements.txt
- main.py: This is the main file which combines all the other scripts to output the required info.
- caption.py: This script contains the model for image captioning and returns image caption for a particular input image.
- summarizer.py: This script contains all the necessary functions to summarize the text obtained from the pdf.
After the requirements have been installed, the process from training to testing is fairly easy. The commands to run:
- Upload the pdf which is required to be summarised in the main directory.
- Update the path of the pdf in the main.py script.
- Update all the paths in the caption.py script from line 40 to 44 accordingly.
- Run the main.py script.
Hurrah!! You have got yourself some notes compiled from provided pdf along with captioned images!