Storage

There are two types of storage.

Raw file storage

Raw file storage stores the response received as-is, this is done automatically with FileManager. You may implement your own raw file storage using the interface:

ai.preferred.venom.storage.FileManager

Otherwise, we provide MysqlFileManager which utilise the local filesystem as storage backed by a MySQL record. this can be found in

ai.preferred.venom.storage.MysqlFileManager

Information storage

Information storage stores the information extracted from a web page and is often structured and clean. For example, the information of this book can be stored in this format:

• Title = "A Light in the Attic"
• Price = 51.77
• Tax = 0.00
• Stock = 0
• UPC = "a897fe39b1053632"

For this purpose, you can use any storage engine (database). Some examples of databases include: MariaDB, MongoDB, MySQL, PostgreSQL. You may also use Object/Relational Mapping (ORM) frameworks such as Java Persistence API or Hibernate to simplify your project.

Check out the Wiki for the respective section for more details