Commit Graph

Select branches

Hide Pull Requests

main

minio

spider-rs

#1

Mono Color

• 946d965d30 update storage shcema minio oliver 2024-12-13 18:02:54 -0700
• 82929fd0fc updating for base64 Oliver Atkinson 2024-12-13 13:28:24 -0700
• f42e770a10 moved to other repo Oliver Atkinson 2024-12-13 11:01:35 -0700
• 611a1e923b starting on the extension Oliver Atkinson 2024-12-12 15:32:04 -0700
• 298ad39a79 rename Oliver Atkinson 2024-12-12 14:59:54 -0700
• 215056e493 use contains operator for better output Oliver Atkinson 2024-12-12 14:26:49 -0700
• 22be3b2f61 updating deps Oliver Atkinson 2024-12-12 14:14:38 -0700
• c1c8cf07bb unifed settings for testing Oliver Atkinson 2024-12-12 11:42:07 -0700
• e4bf320ecd unifed settings for testing spider-rs Oliver Atkinson 2024-12-12 11:41:39 -0700
• fd71a8bc13 back to spider-rs Oliver Atkinson 2024-12-12 11:04:45 -0700
• 0f8a3d7215 using a custom parser now :) oliver 2024-11-12 23:08:09 -0700
• 574a370f30 readme updates oliver 2024-11-12 21:24:57 -0700
• eaa79b749e prepare get function for s3 oliver 2024-11-12 21:19:05 -0700
• 2c28d69d55 add s3 support oliver 2024-11-12 21:03:58 -0700
• d28d18de08 formatting and timer changes, consolidated functions main oliver 2024-11-12 18:40:10 -0700
• 8a5ac61b26 Merge pull request 'custom_engine' (#1) from custom_engine into main Oliver 2024-11-13 00:53:31 +0000
• d9d4c56142 fits with compose.yml oliver 2024-11-12 17:51:13 -0700
• f87a43c3a9 added demo site for testing elements oliver 2024-11-12 17:50:59 -0700
• 7cac880f8e remove un-used function oliver 2024-11-12 17:50:28 -0700
• 720adaa552 added support for nearly all html tags that can have a link oliver 2024-11-12 17:50:06 -0700
• 7c32600694 this is really all that's needed oliver 2024-11-12 17:49:45 -0700
• 399510c599 use reqwest client for epic speedup oliver 2024-11-10 20:37:00 -0700
• ec66c4e765 remove unused import oliver 2024-11-10 20:36:39 -0700
• a9628ee5e4 working, now onto speeding it up oliver 2024-11-10 20:24:04 -0700
• 5404d5c3e8 it works :party: oliver 2024-11-09 23:30:57 -0700
• fd971bafbf it works now oliver 2024-11-09 15:28:10 -0700
• c3997b0bb7 works more, but still not all the way oliver 2024-11-09 11:30:32 -0700
• 7826c4cec6 jank-ish fix but it sure does work make the root record (for links https://example.com/) have a record id of the url, thus preventing duplication when using upsert Oliver Atkinson 2024-10-31 15:32:37 -0600
• 3a46dd937b updates Oliver Atkinson 2024-10-31 15:09:48 -0600
• fbca067b1f clean up walk() Oliver Atkinson 2024-10-31 14:10:14 -0600
• 9324160e74 crawling 🕷️ Oliver Atkinson 2024-10-07 11:14:56 -0600
• 974bccc457 no longer using spider, just wiritng my own crawler Oliver Atkinson 2024-10-04 13:52:34 -0600
• 2d2b09116e lockfile has update 🤷 oliver 2024-08-26 23:15:06 -0600
• 38fe7d3a59 add useage oliver 2024-08-26 01:14:10 -0600
• 6e95aeb154 install instructions oliver 2024-08-26 01:09:46 -0600
• fdbc49337b use custom spider dependency oliver 2024-08-26 01:06:13 -0600
• a507d8dfaa readmd with test results oliver 2024-08-26 01:01:11 -0600
• 2017a08f4c update schema oliver 2024-08-26 00:57:36 -0600
• 67d89ff9eb it helps when you don't reference [0] for two separate elements... oliver 2024-08-26 00:55:36 -0600
• edf9bfc1f5 update needed to merge, which isn't the default action oliver 2024-08-26 00:49:04 -0600
• 409f8c0c01 let me cook! oliver 2024-08-25 15:50:59 -0600
• e66131b411 add oliver 2024-08-23 05:22:49 -0600
• bfed9a6ca6 first commit oliver 2024-08-23 05:21:42 -0600