Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,6 @@ wheels/
.env.*

# Virtual environments
.venv
.venv

uv.lock
23 changes: 6 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ markdown_pages = parser.convert_pdf(pdf_path)
> [!TIP]
> Please refer to [FAQs](docs/faq.md) for more details on how to improve the performance of locally hosted vision models.

### API Models Usage (OpenAI, Azure OpenAI, Gemini, DeepSeek)
### API Models Usage (OpenAI, Azure OpenAI, Gemini)

```python
from vision_parse import VisionParser
Expand Down Expand Up @@ -155,7 +155,7 @@ parser = VisionParser(

# Initialize parser with Google Gemini model
parser = VisionParser(
model_name="gemini-1.5-flash",
model_name="gemini-2.5-flash",
api_key="your-gemini-api-key", # Get the Gemini API key from https://aistudio.google.com/app/apikey
temperature=0.7,
top_p=0.4,
Expand All @@ -164,16 +164,6 @@ parser = VisionParser(
enable_concurrency=True,
)

# Initialize parser with DeepSeek model
parser = VisionParser(
model_name="deepseek-chat",
api_key="your-deepseek-api-key", # Get the DeepSeek API key from https://platform.deepseek.com/api_keys
temperature=0.7,
top_p=0.4,
image_mode="url",
detailed_extraction=False, # Set to True for more detailed extraction
enable_concurrency=True,
)
```

## ✅ Supported Models
Expand All @@ -184,15 +174,14 @@ This package supports the following Vision LLM models:
|:------------:|:----------:|
| gpt-4o | OpenAI |
| gpt-4o-mini | OpenAI |
| gemini-1.5-flash | Google |
| gemini-2.0-flash-exp | Google |
| gemini-1.5-pro | Google |
| gemini-2.5-pro | Google |
| gemini-2.5-flash | Google |
| gemini-2.0-flash | Google |
| llava:13b | Ollama |
| llava:34b | Ollama |
| llama3.2-vision:11b | Ollama |
| llama3.2-vision:70b | Ollama |
| deepseek-r1:32b | Ollama |
| deepseek-chat | DeepSeek |

## 🔧 Customization Parameters

Expand All @@ -203,7 +192,7 @@ Vision Parse offers several customization parameters to enhance document process
| model_name | Name of the Vision LLM model to use | str |
| custom_prompt | Define custom prompt for the model and it will be used as a suffix to the default prompt | str |
| ollama_config | Specify custom configuration for Ollama client initialization | dict |
| openai_config | Specify custom configuration for OpenAI, Azure OpenAI or DeepSeek client initialization | dict |
| openai_config | Specify custom configuration for OpenAI or Azure OpenAI client initialization | dict |
| gemini_config | Specify custom configuration for Gemini client initialization | dict |
| image_mode | Sets the image output format for the model i.e. if you want image url in markdown content or base64 encoded image | str |
| detailed_extraction | Enable advanced content extraction to extract complex information such as LaTeX equations, tables, images, etc. | bool |
Expand Down
68 changes: 0 additions & 68 deletions docs/examples/deepseek_demo.ipynb

This file was deleted.

90 changes: 4 additions & 86 deletions docs/examples/gemini_demo.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -25,92 +25,10 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/Users/arunbrahma/anaconda3/envs/mp/lib/python3.13/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
" from .autonotebook import tqdm as notebook_tqdm\n",
"Converting pages in PDF file into markdown format: 100%|██████████| 1/1 [00:15<00:00, 15.82s/it]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"--- Page 1 ---\n",
"# TEXAS A&M\n",
"# AGRILIFE\n",
"# EXTENSION\n",
"\n",
"March 3, 2020\n",
"\n",
"TO: JR Sprague, TCAAA D1 Chairman\n",
"\n",
"FROM: Andy Holloway, CEA ANR Hemphill County\n",
"\n",
"Regarding the three chairmanships appointed to me none have any action\n",
"needed at this time except for the rules to be submitted to be posted on the\n",
"TCAAA website for the Texas Holdem Tournament. Please note the following:\n",
"\n",
"# Texas Hold'em Basic Poker Rules\n",
"\n",
"Texas Hold'em is played with a standard deck of 52 cards. The object of the game is to make the best\n",
"possible five-card poker hand using any combination of the two card's in a player's hand (known as\n",
"the \"hole cards\") and the five community cards that the dealer places in the middle of the table, which\n",
"can be used by all players. The standard hand rankings are used, as follows (from best to worst):\n",
"\n",
"* Straight Flush: Any five cards of the same suit and consecutive rank; the best of these hands,\n",
"AKQJT of a single suit, is known as a Royal Flush. For these and regular straights, aces may be\n",
"played as high or low cards.\n",
"* Four of a Kind: Any four cards of the same rank.\n",
"* Full House: Any three cards from a single rank combined with a pair from a different rank (i.e.,\n",
"TT55).\n",
"* Flush: Any five cards of the same suit.\n",
"* Straight: Any five cards of consecutive ranks (i.e., 34567).\n",
"* Three of a Kind: Any three cards of the same rank.\n",
"* Two Pair: Any two pairs of cards from the same ranks (ie, 5599A).\n",
"* One Pair: Any two cards of the same rank.\n",
"* High Card: Hands that do not fit any of the above categories are ranked based on the highest card\n",
"in their hand (aces are high), then by the second highest card, and so on.\n",
"\n",
"\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n"
]
}
],
"source": [
"from vision_parse import VisionParser\n",
"import os\n",
"\n",
"# Initialize parser\n",
"parser = VisionParser(\n",
" model_name=\"gemini-1.5-flash\",\n",
" api_key=os.getenv(\"GEMINI_API_KEY\"),\n",
" temperature=0.9,\n",
" top_p=0.4,\n",
" max_output_tokens=2048,\n",
" image_mode=\"url\",\n",
" detailed_extraction=True,\n",
")\n",
"\n",
"pdf_path = \"../tests/Texas-Holdem-Rules.pdf\"\n",
"markdown_pages = parser.convert_pdf(pdf_path)\n",
"\n",
"# Print the markdown pages\n",
"for i, page_content in enumerate(markdown_pages):\n",
" print(f\"\\n--- Page {i+1} ---\\n{page_content}\")"
]
"outputs": [],
"source": "from vision_parse import VisionParser\nimport os\n\n# Initialize parser\nparser = VisionParser(\n model_name=\"gemini-2.5-flash\",\n api_key=os.getenv(\"GEMINI_API_KEY\"),\n temperature=0.9,\n top_p=0.4,\n max_output_tokens=2048,\n image_mode=\"url\",\n detailed_extraction=True,\n)\n\npdf_path = \"../tests/Texas-Holdem-Rules.pdf\"\nmarkdown_pages = parser.convert_pdf(pdf_path)\n\n# Print the markdown pages\nfor i, page_content in enumerate(markdown_pages):\n print(f\"\\n--- Page {i+1} ---\\n{page_content}\")"
}
],
"metadata": {
Expand All @@ -134,4 +52,4 @@
},
"nbformat": 4,
"nbformat_minor": 2
}
}
2 changes: 1 addition & 1 deletion docs/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

This is a known limitation with locally hosted Ollama models. Here are some solutions:

- **Use API-based Models**: For better performance, consider using API-based models like OpenAI, DeepSeek, or Gemini, which are significantly faster and more accurate.
- **Use API-based Models**: For better performance, consider using API-based models like OpenAI or Gemini, which are significantly faster and more accurate.
- **Enable Concurrency**: Set `enable_concurrency` to `True` so that multiple pages are processed in parallel, thereby reducing latency. You can also increase the value of `OLLAMA_NUM_PARALLEL` to maximize the number of pages that can be processed in parallel.
- **Disable Detailed Extraction**: Disable the `detailed_extraction` parameter for simpler PDF documents, which can improve latency.

Expand Down
2 changes: 1 addition & 1 deletion docs/model_config.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Pass the following configuration settings to `ollama_config` parameter while ini

For model-specific parameters (like temperature, top_p, etc.), please refer to the [Ollama Model Parameters documentation](https://github.com/ollama/ollama/blob/main/docs/modelfile.md#valid-parameters-and-values). You can pass model-specific parameters as additional kwargs to the `VisionParser` class.

### OpenAI and DeepSeek Clients:
### OpenAI Client:

Pass the following configuration settings to `openai_config` parameter while initializing the VisionParser class.

Expand Down
7 changes: 4 additions & 3 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,8 @@ dependencies = [
"ollama>=0.4.4",
"opencv-python>=4.10.0.84",
"pydantic>=2.0.0",
"pymupdf>=1.22.0",
"pypdfium2>=4.0.0",
"Pillow>=10.4.0",
"tenacity>=9.0.0",
"tqdm>=4.65.0",
]
Expand All @@ -58,13 +59,13 @@ dev = [
"twine>=6.0.1",
]
gemini = [
"google-generativeai==0.8.3",
"google-genai>=0.2.0",
]
openai = [
"openai==1.59.8",
]
all = [
"google-generativeai==0.8.3",
"google-genai>=0.2.0",
"openai==1.59.8",
]

Expand Down
25 changes: 17 additions & 8 deletions src/vision_parse/constants.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,23 @@

SUPPORTED_MODELS: Dict[str, str] = {
"llama3.2-vision:11b": "ollama",
"llama3.2-vision:70b": "ollama",
"llava:13b": "ollama",
"llava:34b": "ollama",
"deepseek-r1:32b": "ollama",
"gpt-4o": "openai",
"gpt-4o-mini": "openai",
"gemini-1.5-flash": "gemini",
"gemini-2.0-flash-exp": "gemini",
"gemini-1.5-pro": "gemini",
"deepseek-chat": "deepseek",
"gemini-2.5-pro": "gemini",
"gemini-2.5-flash": "gemini",
"gemini-2.0-flash": "gemini",
}


def discover_ollama_vision_models() -> Dict[str, str]:
try:
import ollama
from .model_detector import ModelDetector

client = ollama.Client()
detector = ModelDetector(client)
vision_models = detector.get_vision_models()

return {model: "ollama" for model in vision_models}
except Exception:
return {}
Loading