Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .git-rewrite/backup-refs
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
c2ce8778ef60f973d3bacbaf5f5d56eb08e3192a commit refs/heads/Branch_1_invoice_extractor
510660f659b5ccd743664b5d6f5be5672a6a9eb7 commit refs/heads/main
d471f72947540ce786e5b8d5dd55478b659d5d04 commit refs/heads/master
c2ce8778ef60f973d3bacbaf5f5d56eb08e3192a commit refs/remotes/origin/Branch_1_invoice_extractor
5f3d49b0d1fd84f7acd541cac7eed889dfae3b5e commit refs/remotes/origin/HEAD
c55ff1cc2d28dd2aeabb9c5380a521fb00fc1aec commit refs/remotes/origin/backup_branch
5f3d49b0d1fd84f7acd541cac7eed889dfae3b5e commit refs/remotes/origin/main
d471f72947540ce786e5b8d5dd55478b659d5d04 commit refs/remotes/origin/master
6 changes: 6 additions & 0 deletions .git-rewrite/commit
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
tree 7bb340cd458136db0e1b05615c52997445a513da
parent 6e1b3556db95e113acaeb21c7a51b51f87a66718
author Deep7285 <deepak72855@gmail.com> 1755768164 +0530
committer Deep7285 <deepak72855@gmail.com> 1755768164 +0530

added html file
1 change: 1 addition & 0 deletions .git-rewrite/heads
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
refs/heads/main
1 change: 1 addition & 0 deletions .git-rewrite/map/6e1b3556db95e113acaeb21c7a51b51f87a66718
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
6e1b3556db95e113acaeb21c7a51b51f87a66718
1 change: 1 addition & 0 deletions .git-rewrite/message
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
commit html file
1 change: 1 addition & 0 deletions .git-rewrite/parse
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
510660f659b5ccd743664b5d6f5be5672a6a9eb7
1 change: 1 addition & 0 deletions .git-rewrite/raw-refs
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
refs/heads/main
48 changes: 48 additions & 0 deletions .git-rewrite/revs
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
6e1b3556db95e113acaeb21c7a51b51f87a66718
4056fbc5a750ddc7291c3f0401ef1fa1e4f582b6 6e1b3556db95e113acaeb21c7a51b51f87a66718
905ec53b654412dedaf543f2b06e826e49851bc1 4056fbc5a750ddc7291c3f0401ef1fa1e4f582b6
eac51535f7227fa781f87f1be9b7651ed511757f 905ec53b654412dedaf543f2b06e826e49851bc1
0d55648c0ac61415c5e7d88b99900fe8c9e17ecc eac51535f7227fa781f87f1be9b7651ed511757f
be005b4e651beefcafea27c930a979a01fe64faa 0d55648c0ac61415c5e7d88b99900fe8c9e17ecc
131f0938dfbf45cf4484259688de9fc4c98e3c46 be005b4e651beefcafea27c930a979a01fe64faa
b94415b35dc11019c4e4ac085f4a77ea07c99ed8 131f0938dfbf45cf4484259688de9fc4c98e3c46
87da215f1b059d69b4cef624ff88f483a617d5ea b94415b35dc11019c4e4ac085f4a77ea07c99ed8
87a64dcbba4d9c7e3914c64d25aee32d6e012d29 87da215f1b059d69b4cef624ff88f483a617d5ea
b86c0095c51609bf5227a638364b67c12cd13eb4 87a64dcbba4d9c7e3914c64d25aee32d6e012d29
41bd8e1e193ff4d3f0627b2fb78ac52a5ad4e260 b86c0095c51609bf5227a638364b67c12cd13eb4
89400bf275649522f0ed5bdd4a97b955cf0c8ddc 41bd8e1e193ff4d3f0627b2fb78ac52a5ad4e260
0acf24b0d9c98591d3789ecfdb62cf81e8ee5a5e 89400bf275649522f0ed5bdd4a97b955cf0c8ddc
0ccfbf45b6f94192e31ff84802bedbaeb7042b4e 0acf24b0d9c98591d3789ecfdb62cf81e8ee5a5e
d3777c0c08fef40f39bba4777efe8d468f86d69f 0ccfbf45b6f94192e31ff84802bedbaeb7042b4e
b7b57ed85579ac255deeccb7f7ee227f90d72aaf d3777c0c08fef40f39bba4777efe8d468f86d69f
c4394878a54d182aee6d5a368b14fc83270056b8 b7b57ed85579ac255deeccb7f7ee227f90d72aaf
357c3befeeedc435645ef21f84ea5fd1381b2ac8 c4394878a54d182aee6d5a368b14fc83270056b8
229bef0dd78b42db40dd4834721c619ffcdf8381 357c3befeeedc435645ef21f84ea5fd1381b2ac8
28e6f4adf12c828f358e4be232cac151f29fa1ef 229bef0dd78b42db40dd4834721c619ffcdf8381
cd407424195243bd9035043ec9174155cb152f40 28e6f4adf12c828f358e4be232cac151f29fa1ef
9c234491ff68c31eb0b869b73322806f79189dcd cd407424195243bd9035043ec9174155cb152f40
3f6a1ffb18bec9ead80557c0861d40438f01605e 9c234491ff68c31eb0b869b73322806f79189dcd
c2ce8778ef60f973d3bacbaf5f5d56eb08e3192a 3f6a1ffb18bec9ead80557c0861d40438f01605e
3193107f7e02dab275ff4f364e83b924099130ed c2ce8778ef60f973d3bacbaf5f5d56eb08e3192a
1c315ec9390ed25ac199b130199c40a91501a906 3193107f7e02dab275ff4f364e83b924099130ed
d104872fb96262839021ff8a6e267f72027dea98 3193107f7e02dab275ff4f364e83b924099130ed
11461affcc4524b0275018ad6300265f068ef6ef d104872fb96262839021ff8a6e267f72027dea98
d6499003e21d3cada0326b5cdb817ec7b9bf513b 1c315ec9390ed25ac199b130199c40a91501a906 11461affcc4524b0275018ad6300265f068ef6ef
41b2d4900bc3618ddee201214ae461cb08e37aba d6499003e21d3cada0326b5cdb817ec7b9bf513b
ed1439378fe8a86d1a88910943e2776bc6c15ec0 41b2d4900bc3618ddee201214ae461cb08e37aba
03ffb5db0b0d6042eb7528f07d08f5c97fe8cddc ed1439378fe8a86d1a88910943e2776bc6c15ec0
1d18bde29e69a830617e761f9f9ca9eebfbe39b7 03ffb5db0b0d6042eb7528f07d08f5c97fe8cddc
45642f1aa0bfc6810496f023241ec352a4ccfb7a 1d18bde29e69a830617e761f9f9ca9eebfbe39b7
089c20acf7b08f22191898bff08fc53d2b15328a 45642f1aa0bfc6810496f023241ec352a4ccfb7a
386657fd06198f625a1130d0cf30cfa6871565ac 089c20acf7b08f22191898bff08fc53d2b15328a
df23e6277538f40858823906520a774af14a0611 386657fd06198f625a1130d0cf30cfa6871565ac
5e615511d3cde4edfefd9cd1c99bff34a3eb1f47 df23e6277538f40858823906520a774af14a0611
12a61dd091fbd5d00f6b997797e3731ce7cdb906 5e615511d3cde4edfefd9cd1c99bff34a3eb1f47
5d5039fed82e591b7d06c1ebe6a4faa6851ad409 12a61dd091fbd5d00f6b997797e3731ce7cdb906
c55ff1cc2d28dd2aeabb9c5380a521fb00fc1aec 5d5039fed82e591b7d06c1ebe6a4faa6851ad409
8b43d2e0a893a51f37a1f592d14800849c6b039a c55ff1cc2d28dd2aeabb9c5380a521fb00fc1aec
506a586c0e31ab9731c6fd828f56c085d2dcc7ca 8b43d2e0a893a51f37a1f592d14800849c6b039a
a7433b3d45407f5813d024c9ddb9aac618356229 506a586c0e31ab9731c6fd828f56c085d2dcc7ca
c79d87b24403c317992333a91429abd390231b86 a7433b3d45407f5813d024c9ddb9aac618356229
cf941a555c3a410d8f6b170d6b49a39174e38241 c79d87b24403c317992333a91429abd390231b86
510660f659b5ccd743664b5d6f5be5672a6a9eb7 cf941a555c3a410d8f6b170d6b49a39174e38241
21 changes: 21 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2025 Deepak (Deep7285)

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
3 changes: 2 additions & 1 deletion add-user.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,8 @@ Write-Host "JSON generated OK" -ForegroundColor Green

# Step 2: Write JSON to a temp file (avoids all PowerShell quoting issues)
$tmpFile = [System.IO.Path]::GetTempFileName()
[System.IO.File]::WriteAllText($tmpFile, $jsonLine, [System.Text.Encoding]::UTF8)
$utf8NoBom = New-Object System.Text.UTF8Encoding $false
[System.IO.File]::WriteAllText($tmpFile, $jsonLine, $utf8NoBom)

Write-Host "Temp file: $tmpFile"

Expand Down
38 changes: 31 additions & 7 deletions frontend/extractor-client.js
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@ let isRunning = false; // prevent double-submit
function getSessionUser() { return sessionStorage.getItem("dx_user"); }
function setSessionUser(u) { sessionStorage.setItem("dx_user", u); }
function clearSessionUser() { sessionStorage.removeItem("dx_user"); }
function getSessionToken() { return sessionStorage.getItem("dx_token"); }
function setSessionToken(t) { sessionStorage.setItem("dx_token", t); }
function clearSessionToken() { sessionStorage.removeItem("dx_token"); }

// ── Toast notifications ───────────────────────────────────────────────────────
function showToast(message, type = "info", duration = 4000) {
Expand Down Expand Up @@ -94,11 +97,14 @@ async function apiLogin(username, password, honeypot) {

// ── API: Logout ───────────────────────────────────────────────────────────────
async function apiLogout() {
const token = getSessionToken();
await fetch(`${WORKER_ENDPOINT}/api/logout`, {
method: "POST",
credentials: "include"
credentials: "include",
headers: token ? { "Authorization": `Bearer ${token}` } : {}
}).catch(() => {});
clearSessionUser();
clearSessionToken();
syncAuthUI();
showToast("Logged out successfully", "info");
}
Expand Down Expand Up @@ -153,6 +159,7 @@ async function handleLoginSubmit() {
try {
const data = await apiLogin(username, password, honeypot);
setSessionUser(data.username);
if (data.token) setSessionToken(data.token);
syncAuthUI();
closeLoginModal();
showToast(`Welcome back, ${data.username}!`, "success");
Expand Down Expand Up @@ -240,9 +247,11 @@ async function postToWorker({ imagesDataUrls = [], docText = "" }) {
for (const d of imagesDataUrls) form.append("images_dataurl[]", d);
if (docText?.trim()) form.append("doc_text", docText.trim());

const token = getSessionToken();
const resp = await fetch(`${WORKER_ENDPOINT}/api/extract`, {
method: "POST",
credentials: "include", // ← CRITICAL: session cookie must travel with request
credentials: "include",
headers: token ? { "Authorization": `Bearer ${token}` } : {},
body: form
});

Expand Down Expand Up @@ -320,11 +329,26 @@ function renderTable(rows) {
// ── Download Excel ────────────────────────────────────────────────────────────
function downloadExcel() {
if (!allRows.length) { showToast("Nothing to download yet.", "info"); return; }
const ws = XLSX.utils.json_to_sheet(allRows);
const wb = XLSX.utils.book_new();
XLSX.utils.book_append_sheet(wb, ws, "Invoices");
XLSX.writeFile(wb, `invoices_${Date.now()}.xlsx`);
showToast("Excel file downloaded!", "success");
try {
if (typeof XLSX === "undefined") throw new Error("XLSX library not loaded. Check your internet connection and reload the page.");
const ws = XLSX.utils.json_to_sheet(allRows);
const wb = XLSX.utils.book_new();
XLSX.utils.book_append_sheet(wb, ws, "Invoices");
const buf = XLSX.write(wb, { bookType: "xlsx", type: "array" });
const blob = new Blob([buf], { type: "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet" });
const url = URL.createObjectURL(blob);
const a = document.createElement("a");
a.href = url;
a.download = `invoices_${Date.now()}.xlsx`;
document.body.appendChild(a);
a.click();
document.body.removeChild(a);
setTimeout(() => URL.revokeObjectURL(url), 1000);
showToast("Excel file downloaded!", "success");
} catch (e) {
showToast("Download failed: " + e.message, "error");
console.error("[DataExtract] Excel download error:", e);
}
}

// ── Reset session ─────────────────────────────────────────────────────────────
Expand Down
22 changes: 22 additions & 0 deletions frontend/xlsx.full.min.js

Large diffs are not rendered by default.

47 changes: 41 additions & 6 deletions index.html
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
<script src="https://cdn.jsdelivr.net/npm/pdfjs-dist@3.11.174/build/pdf.min.js"></script>
<script>pdfjsLib.GlobalWorkerOptions.workerSrc="https://cdn.jsdelivr.net/npm/pdfjs-dist@3.11.174/build/pdf.worker.min.js";</script>
<script src="https://unpkg.com/mammoth/mammoth.browser.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/xlsx/0.19.3/xlsx.full.min.js" crossorigin="anonymous"></script>
<script src="frontend/xlsx.full.min.js"></script>

<style>
/* Light theme (default) */
Expand Down Expand Up @@ -459,6 +459,15 @@
}
.field input:focus { border-color: var(--accent); box-shadow: 0 0 0 3px var(--accent-dim); }
.field-hp { display: none !important; }
.field-pw-wrap { position: relative; }
.field-pw-wrap input { padding-right: 42px; }
.btn-pw-toggle {
position: absolute; right: 10px; top: 50%; transform: translateY(-50%);
background: none; border: none; cursor: pointer; color: var(--text-muted);
padding: 4px; display: flex; align-items: center; transition: color .15s;
}
.btn-pw-toggle:hover { color: var(--text-dim); }
.btn-pw-toggle svg { width: 18px; height: 18px; }
.login-err { background: rgba(232,70,70,.1); border: 1px solid rgba(232,70,70,.25); border-radius: 8px; padding: 10px 14px; font-size: 13px; color: #EF8888; margin-bottom: 16px; }
.modal-acts { display: flex; gap: 10px; justify-content: flex-end; margin-top: 8px; }
.btn-cancel { padding: 10px 20px; background: transparent; color: var(--text-dim); border: 1px solid var(--border-hi); border-radius: 10px; font-weight: 600; cursor: pointer; font-size: 14px; }
Expand Down Expand Up @@ -493,11 +502,13 @@
.foot-links { display: flex; align-items: center; gap: 10px; }
.foot-link {
display: inline-flex; align-items: center; gap: 7px;
padding: 7px 14px; background: var(--surface-2);
border: 1px solid var(--border); border-radius: 8px;
font-size: 13px; font-weight: 600; color: var(--text-dim); transition: all .2s var(--ease);
padding: 7px 16px; background: var(--surface-2);
border: 1px solid var(--accent); border-radius: 8px;
font-size: 13px; font-weight: 600; color: var(--accent);
box-shadow: 0 0 0 3px var(--accent-dim);
transition: all .2s var(--ease);
}
.foot-link:hover { color: var(--text); border-color: var(--border-hi); text-decoration: none; }
.foot-link:hover { background: var(--accent-dim); border-color: var(--accent); color: var(--accent); box-shadow: var(--acc-glow); text-decoration: none; }

.btn-theme {
width: 36px; height: 36px; border-radius: 8px;
Expand Down Expand Up @@ -741,7 +752,14 @@ <h2 class="sec-h">Three steps, zero friction</h2>
</div>
<div class="field">
<label for="loginPassword">Password</label>
<input type="password" id="loginPassword" autocomplete="current-password" placeholder="Your password" />
<div class="field-pw-wrap">
<input type="password" id="loginPassword" autocomplete="current-password" placeholder="Your password" />
<button type="button" class="btn-pw-toggle" id="pwToggle" aria-label="Show password">
<svg id="eyeIcon" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path d="M1 12s4-8 11-8 11 8 11 8-4 8-11 8-11-8-11-8z"/><circle cx="12" cy="12" r="3"/>
</svg>
</button>
</div>
</div>
<!-- Honeypot trap — invisible to humans -->
<div class="field field-hp" aria-hidden="true">
Expand Down Expand Up @@ -774,5 +792,22 @@ <h2 class="sec-h">Three steps, zero friction</h2>
})();
</script>
<script src="frontend/extractor-client.js"></script>
<script>
(function(){
var toggle = document.getElementById('pwToggle');
var input = document.getElementById('loginPassword');
var icon = document.getElementById('eyeIcon');
var eyeOpen = '<path d="M1 12s4-8 11-8 11 8 11 8-4 8-11 8-11-8-11-8z"/><circle cx="12" cy="12" r="3"/>';
var eyeSlash = '<path d="M17.94 17.94A10.07 10.07 0 0 1 12 20c-7 0-11-8-11-8a18.45 18.45 0 0 1 5.06-5.94M9.9 4.24A9.12 9.12 0 0 1 12 4c7 0 11 8 11 8a18.5 18.5 0 0 1-2.16 3.19m-6.72-1.07a3 3 0 1 1-4.24-4.24"/><line x1="1" y1="1" x2="23" y2="23"/>';
if (toggle && input && icon) {
toggle.addEventListener('click', function(){
var show = input.type === 'password';
input.type = show ? 'text' : 'password';
icon.innerHTML = show ? eyeSlash : eyeOpen;
toggle.setAttribute('aria-label', show ? 'Hide password' : 'Show password');
});
}
})();
</script>
</body>
</html>
116 changes: 116 additions & 0 deletions readme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
# DataExtract: An AI Invoice Data Extractor

A serverless web app that reads invoices, printed, scanned, or handwritten and exports the structured data to Excel.
---

## What it does

Drop in a PDF, image (jpg/png), or Word document. The app sends it to an AI vision model, pulls out the relevant fields, and hands you back a formatted Excel file. The file never touches a database, it's read in memory and discarded the moment extraction is done.

It handles digital PDFs, scanned images, and handwritten bills — including ones in Hindi script, which was a real requirement from the client.

---

## Why it was built this way

The client had two hard constraints: no third-party file storage for compliance reasons, and the monthly cost had to stay near zero for a small team of users.

The solution was to process everything at the edge with Cloudflare Workers and never write files to disk. The frontend converts PDFs to images in the browser using pdf.js, compresses them, and posts them directly to the worker. The worker calls the AI model, gets back structured JSON, and returns it.

Cloudflare's free tier handles the worker and the KV store (used for user auth). GitHub Pages hosts the frontend.
---

## Architecture

```
Browser (GitHub Pages)
│ POST /api/extract (FormData: JPEG data URLs)
Cloudflare Worker (KV: session store + user auth)
│ generateContent (vision model)
Google Gemini Flash API
└─► JSON → back to browser → rendered as table → Excel download
```

The worker also handles login/logout. Sessions are stored in Cloudflare KV with a 30-day TTL. Passwords are hashed with PBKDF2-SHA256. Unauthenticated users get 2 free extraction sessions; after that, they're prompted to log in.

---

## Running locally

You need Node.js 18+ and a Cloudflare account.

```bash
git clone https://github.com/Deep7285/data-extractor.git
cd data-extractor

npm install

# Create your local environment file
cp .dev.vars.example .dev.vars
# Add your GEMINI_API_KEY inside .dev.vars

# Start the Cloudflare Worker locally
npx wrangler dev

# Open index.html directly in a browser (or use a local dev server)
```

The frontend is plain HTML/CSS/JS — no build step, no framework.

---

## Environment variables

These go in `.dev.vars` for local development, and as secrets in the Cloudflare dashboard for production. Do not commit this file.

| Variable | Purpose |
|---|---|
| `GEMINI_API_KEY` | Google Gemini API key (get one free at aistudio.google.com) |

---

## Adding users

Users are stored directly in Cloudflare KV. To add one:

```bash
# 1. Generate a password hash using the make-user tool in /tools
node tools/make-user.mjs

# 2. Copy the output and store it in KV
wrangler kv:key put --binding=USERS user:deepak '{"username":"deepak",...}'
```

Passwords are never stored in plain text. The hash uses PBKDF2-SHA256 with a random salt and is verified in the worker on each login attempt.

---

## What's not in this repo

The Cloudflare Worker source (`src/`) is kept in a separate private repository. What's here is the frontend only. The backend handles all auth, API calls, and rate limiting — the frontend has no API keys and no secrets.

If you're a developer looking to understand the architecture or discuss implementation, feel free to reach out directly.

---

## Stack

- Vanilla HTML/CSS/JavaScript
- Cloudflare Workers (serverless edge backend)
- Cloudflare KV (user + session store)
- Google Gemini Flash (vision OCR)
- pdf.js (client-side PDF to image conversion)
- Mammoth.js (DOCX text extraction)
- SheetJS (Excel generation in the browser)
- Wrangler CLI (Cloudflare local dev + deploy)

---

## Contact

Built by Deepak Kumar (Deep7285)
Loading
Loading