|
|
d28d18de08
|
formatting and timer changes, consolidated functions
|
2024-11-12 18:40:10 -07:00 |
|
|
|
8a5ac61b26
|
Merge pull request 'custom_engine' (#1) from custom_engine into main
Reviewed-on: #1
|
2024-11-13 00:53:31 +00:00 |
|
|
|
d9d4c56142
|
fits with compose.yml
|
2024-11-12 17:51:13 -07:00 |
|
|
|
f87a43c3a9
|
added demo site for testing elements
|
2024-11-12 17:50:59 -07:00 |
|
|
|
7cac880f8e
|
remove un-used function
|
2024-11-12 17:50:28 -07:00 |
|
|
|
720adaa552
|
added support for nearly all html tags that can have a link
|
2024-11-12 17:50:06 -07:00 |
|
|
|
7c32600694
|
this is really all that's needed
|
2024-11-12 17:49:45 -07:00 |
|
|
|
399510c599
|
use reqwest client for epic speedup
|
2024-11-10 20:37:00 -07:00 |
|
|
|
ec66c4e765
|
remove unused import
|
2024-11-10 20:36:39 -07:00 |
|
|
|
a9628ee5e4
|
working, now onto speeding it up
|
2024-11-10 20:24:04 -07:00 |
|
|
|
5404d5c3e8
|
it works :party:
|
2024-11-09 23:30:57 -07:00 |
|
|
|
fd971bafbf
|
it works now
|
2024-11-09 15:28:10 -07:00 |
|
|
|
c3997b0bb7
|
works more, but still not all the way
|
2024-11-09 11:30:32 -07:00 |
|
Oliver Atkinson
|
7826c4cec6
|
jank-ish fix but it sure does work
make the root record (for links https://example.com/) have a record id of the url, thus preventing duplication when using upsert
|
2024-10-31 15:32:37 -06:00 |
|
Oliver Atkinson
|
3a46dd937b
|
updates
|
2024-10-31 15:09:48 -06:00 |
|
Oliver Atkinson
|
fbca067b1f
|
clean up walk()
|
2024-10-31 14:10:14 -06:00 |
|
Oliver Atkinson
|
9324160e74
|
crawling 🕷️
|
2024-10-07 11:14:56 -06:00 |
|
Oliver Atkinson
|
974bccc457
|
no longer using spider, just wiritng my own crawler
|
2024-10-04 13:52:34 -06:00 |
|
|
|
2d2b09116e
|
lockfile has update 🤷
|
2024-08-26 23:15:06 -06:00 |
|
|
|
38fe7d3a59
|
add useage
|
2024-08-26 01:14:10 -06:00 |
|
|
|
6e95aeb154
|
install instructions
|
2024-08-26 01:09:46 -06:00 |
|
|
|
fdbc49337b
|
use custom spider dependency
|
2024-08-26 01:06:13 -06:00 |
|
|
|
a507d8dfaa
|
readmd with test results
|
2024-08-26 01:01:11 -06:00 |
|
|
|
2017a08f4c
|
update schema
|
2024-08-26 00:57:36 -06:00 |
|
|
|
67d89ff9eb
|
it helps when you don't reference [0] for two separate elements...
|
2024-08-26 00:55:36 -06:00 |
|
|
|
edf9bfc1f5
|
update needed to merge, which isn't the default action
|
2024-08-26 00:49:04 -06:00 |
|
|
|
409f8c0c01
|
let me cook!
|
2024-08-25 15:50:59 -06:00 |
|
|
|
e66131b411
|
add
|
2024-08-23 05:22:49 -06:00 |
|
|
|
bfed9a6ca6
|
first commit
|
2024-08-23 05:21:42 -06:00 |
|
