13 commits

Author	SHA1	Message	Date
cynthia	8ef1bb0910	categorize simplex links and tor onion links	2025-04-18 10:04:19 +01:00
cynthia	bf40730292	make special case for simplex links	2025-04-15 00:34:57 +01:00
cynthia	e7abd25857	switch instancepath to rootpath	2025-04-05 15:32:01 +00:00
cynthia	63d89b9b8b	add crawler dir for default crawler outputs	2025-04-05 14:41:26 +00:00
cynthia	e76d29807d	make the verified csv file optional argument	2025-04-05 12:15:49 +00:00
cynthia	323a47dd46	fix blacklist matching	2025-04-05 11:30:50 +00:00
cynthia	e764c5e6f3	add new col for hostname, check for scheme and add http:// by default	2025-04-05 11:24:35 +00:00
cynthia	c6518363c1	fix blacklist	2025-04-05 08:56:35 +00:00
cynthia	fe6b826027	add blacklist	2025-04-05 00:27:54 +00:00
cynthia	c041e5df19	replace default parameters with paths to the current lantern instance paths	2025-04-05 00:00:47 +00:00
cynthia	6cc297fe66	add dotenv	2025-04-04 18:10:35 +00:00
cynthia	08e2b5ab20	switch to IsUrlValid	2025-04-03 21:53:37 +00:00
cynthia	0a0233a8a2	add crawler.py	2025-04-03 16:43:22 +00:00