darknet-lantern/README.md

Darknet Lantern Project (WIP)

TODOLIST: 95% completed

# Main features:
DONE:
-py : option 6) Trust/Untrust/Blacklist a webring participant
-php : make a search engine prompt that only accepts [a-zA-Z.://], it must refuse every other character 
-py : fix uptimecheck.py to match the new csv format 
-php : if valid make it filter your own verified.csv and unverified.csv files
-py : option 9)  cleanup all duplicates in your own unverified.csv and verified.csv
-py : option 10) perform sanity checks on all csv files (to mark them as sensitive or remove the ones that are blacklisted)
-py : option 7) Add/Remove words in the sensitive list			
-py : option 8) Add/Remove words in the blacklist				
-manual work: fit all the existing links into the current format one by one
-php/css: make the search page preety

TODO:
-doc: redo the documentation for the project
-doc: finish the blogpost about it
-release it officially

What is the Darknet Lantern Project ?

Darknet Lantern is a minimalistic Peer to Peer Decentralised Search Engine for the Darknet.

Why is the Darknet Lantern relevant ?

The Darknet is fundamentally different compared to the Clearnet. Where the Clearnet is like an open city where every destination is visible from the sky, where every website is indexable and searchable using popular search engines, the Darknet is like an immense, dense dark forest where you can't see any destination from the sky.

To explore the Darknet, we need peers (like you and me) to maintain their own lists of onion links for each other, to be able to know where to go, as search engines can't automatically crawl onion websites like they do on the clearnet. The Darknet Lantern Project is a serious attempt at solving the fundamental lack of search engines on the Darknet by organising the manual peer-based work of discovering hidden services in the most efficient way, to make sure Hidden Websites can get visibility while maintaining their anonymity.

Why is the Darknet Lantern Webring relevant ?

The Darknet Lantern Project includes a Webring, as otherwise the Darknet Lanterns would remain scattered and remain hidden from each other, to participate in the webring is to exponentially increase the visibility of the Onion websites you listed, the more webring participants there are.

The more people join the Darknet Lantern Webring, the more visibility Darknet websites will be able to have over time, just like how it would be on the clearnet, while keeping the decentralisation and censorship-resistance intact. All you need is to run your own Darknet Lantern instance, and to join the Webring as a new participant.

How can I setup my own Darknet Lantern ?

We'll first cover how you can run your own Darknet Lantern instance, In short you'll need the latest debian (which is debian 12 currently), Tor, php8.2-fpm, python and a few python libraries that you'll install via the apt package manager.

0. git clone the repository in your directory of choice:

[ Wonderland ] [ /dev/pts/23 ] [/srv/darknet-lantern]
→ apt install tor git torsocks -y

[ Wonderland ] [ /dev/pts/23 ] [/srv/darknet-lantern]
→ torsocks git clone http://git.nowherejezfoltodf4jiyl6r56jnzintap5vyjlia7fkirfsnfizflqd.onion/nihilist/darknet-lantern /srv/darknet-lantern

1. Install nginx and php8.2-fpm

[ Wonderland ] [ /dev/pts/23 ] [/srv/darknet-lantern]
→ apt install php8.2-fpm nginx -y

2. use the nginx.conf and drop it in /etc/nginx/sites-available/

[ Wonderland ] [ /dev/pts/23 ] [/srv/darknet-lantern]
→ ls
nginx.conf  README.md  scripts  todo.txt  torrc  www

[ Wonderland ] [ /dev/pts/23 ] [/srv/darknet-lantern]
→ cp nginx.conf /etc/nginx/sites-available/lantern.conf

[ Wonderland ] [ /dev/pts/23 ] [/srv/darknet-lantern]
→ vim /etc/nginx/sites-available/lantern.conf

3. use mkp244o if you want to have a custom vanity v3 hidden service domain name

4. use the torrc config to have a local socks5 port (as it will be used by the python script to check the uptime of the listed onion links)

[ Wonderland ] [ /dev/pts/23 ] [/srv/darknet-lantern]
→ vim /etc/tor/torrc

[ Wonderland ] [ /dev/pts/23 ] [/srv/darknet-lantern]
→ cat /etc/tor/torrc

HiddenServiceDir /var/lib/tor/onions/nowherejezfoltodf4jiyl6r56jnzintap5vyjlia7fkirfsnfizflqd.onion/
HiddenServicePort 80 127.0.0.1:4443
SocksPort 127.0.0.1:9050

[ Wonderland ] [ /dev/pts/23 ] [/srv/darknet-lantern]
→ systemctl restart tor@default

5. enable the nginx config and validate that it can be accessed on Tor

[ Wonderland ] [ /dev/pts/23 ] [/srv/darknet-lantern]
→ ln -s /etc/nginx/sites-available/lantern.conf /etc/nginx/sites-enabled/

[ Wonderland ] [ /dev/pts/23 ] [/srv/darknet-lantern]
→ nginx -s reload

6) install the python script dependencies:
```sh
[ Wonderland ] [ /dev/pts/23 ] [/srv/darknet-lantern]
→ apt install python3-pandas python3-requests python3-socks

Now that's done, you can run scripts/lantern.py for the first time to confirm your own instance name:

[ Wonderland ] [ /dev/pts/20 ] [/srv/darknet-lantern]
→ python3 scripts/lantern.py
[+] Instance Path doesn't exist yet
What is your Instance domain ? (ex: uptime.nowherejezfoltodf4jiyl6r56jnzintap5vyjlia7fkirfsnfizflqd.onion): lantern.nowherejezfoltodf4jiyl6r56jnzintap5vyjlia7fkirfsnfizflqd.onion

[+] Instance Name:  lantern.nowherejezfoltodf4jiyl6r56jnzintap5vyjlia7fkirfsnfizflqd.onion True

lantern.nowherejezfoltodf4jiyl6r56jnzintap5vyjlia7fkirfsnfizflqd.onion
Is this your this your instance domain ? (y/n)y
OK writing the instance url to ~/.darknet_participants_url
[+] file written, let's read it
lantern.nowherejezfoltodf4jiyl6r56jnzintap5vyjlia7fkirfsnfizflqd.onion
[+] Initial Setup Completed!
[+] file exists, your Webring URL is lantern.nowherejezfoltodf4jiyl6r56jnzintap5vyjlia7fkirfsnfizflqd.onion

In another terminal you can validate that ~/darknet_participant_url has been created properly:

[ Wonderland ] [ /dev/pts/33 ] [/srv/darknet-lantern]
→ cat ~/.darknet_participant_url
lantern.nowherejezfoltodf4jiyl6r56jnzintap5vyjlia7fkirfsnfizflqd.onion#

This is the file that lantern.py will check to get your instance name the next times you run the script. Next you can run scripts/lantern.py again to start adding websites:

[ Wonderland ] [ /dev/pts/20 ] [/srv/darknet-lantern]
→ python3 scripts/lantern.py
[+] Instance Name: lantern.nowherejezfoltodf4jiyl6r56jnzintap5vyjlia7fkirfsnfizflqd.onion True
[+] file exists, your Webring URL is lantern.nowherejezfoltodf4jiyl6r56jnzintap5vyjlia7fkirfsnfizflqd.onion





[+] Welcome to your own Darknet Lantern Instance, where you can explore the Darknet and help others do the same.

Managing Websites:
 1) Add a new Website entry (into unverified.csv)
 2) Trust a Website entry (move an entry from unverified to verified.csv)
 3) Untrust a Website entry (move an entry from unverified to verified.csv)

Managing Webring Participants:
 4) Synchronize new links from existing webring participants, into your unverified.csv file
 5) Add a new webring participant (and download their files into their directory (without trusting them yet!))
 6) Trust/UnTrust/Blacklist a webring participant (Potentially dangerous)

Managing Wordlists:
 7) Add/Remove Words/URLs in the sensitive list (ex: drug)
 8) Add/Remove Words/URLs or links in the blacklist (ex: porn)

Maintenance:
 9) Remove the duplicate URLs for your own instance
 10) Perform sanity checks on all csv files for all instances (to mark them as sensitive / or remove the ones that are blacklisted)

 0) Exit

Select Option? (0-11): 

Here we select option 1 to add some websites:

Select Option? (0-11): 1
1

[+] Add a new Website entry (into unverified.csv)
What is the Website name ? Qubes OS Website
What is the website Category ? Tools
What is the website URL ? http://qubesosfasa4zl44o4tws22di6kepyzfeqv3tg4e3ztknltfxqrymdad.onion/
Description for the website ? (Optional) OS based on Xen that focuses on compartmentalization and virtualization.
Is the website sensitive ? (ex: related to drugs) (y/n) n
[+] NEWROW= ['lantern.nowherejezfoltodf4jiyl6r56jnzintap5vyjlia7fkirfsnfizflqd.onion', 'Tools', 'Qubes OS Website', 'http://qubesosfasa4zl44o4tws22di6kepyzfeqv3tg4e3ztknltfxqrymdad.onion/', '❌', 'OS based on Xen that focuses on compartmentalization and virtualization.', '', '']
[+] New row added! now writing the csv file:

[+] Want to add another website ? (y/n) y

[+] Add a new Website entry (into unverified.csv)
What is the Website name ? Whonix Website
What is the website Category ? Tools
What is the website URL ? http://www.dds6qkxpwdeubwucdiaord2xgbbeyds25rbsgr73tbfpqpt4a6vjwsyd.onion/
Description for the website ? (Optional) VM for general anonymous use
Is the website sensitive ? (ex: related to drugs) (y/n) n
[+] NEWROW= ['lantern.nowherejezfoltodf4jiyl6r56jnzintap5vyjlia7fkirfsnfizflqd.onion', 'Tools', 'Whonix Website', 'http://www.dds6qkxpwdeubwucdiaord2xgbbeyds25rbsgr73tbfpqpt4a6vjwsyd.onion/', '❌', 'VM for general anonymous use', '', '']
[+] New row added! now writing the csv file:

[+] Want to add another website ? (y/n) y

[+] Add a new Website entry (into unverified.csv)
What is the Website name ? Feather Wallet
What is the website Category ? Tools
What is the website URL ? http://featherdvtpi7ckdbkb2yxjfwx3oyvr3xjz3oo4rszylfzjdg6pbm3id.onion/
Description for the website ? (Optional) Lightweight Monero Wallet
Is the website sensitive ? (ex: related to drugs) (y/n) n
[+] NEWROW= ['lantern.nowherejezfoltodf4jiyl6r56jnzintap5vyjlia7fkirfsnfizflqd.onion', 'Tools', 'Feather Wallet', 'http://featherdvtpi7ckdbkb2yxjfwx3oyvr3xjz3oo4rszylfzjdg6pbm3id.onion/', '❌', 'Lightweight Monero Wallet', '', '']
[+] New row added! now writing the csv file:

[+] Want to add another website ? (y/n) y

[+] Add a new Website entry (into unverified.csv)
What is the Website name ? Dark Forest
What is the website Category ? Forums
What is the website URL ? http://dkforestseeaaq2dqz2uflmlsybvnq2irzn4ygyvu53oazyorednviid.onion/
Description for the website ? (Optional)
Is the website sensitive ? (ex: related to drugs) (y/n) y
[+] NEWROW= ['lantern.nowherejezfoltodf4jiyl6r56jnzintap5vyjlia7fkirfsnfizflqd.onion', 'Forums', 'Dark Forest', 'http://dkforestseeaaq2dqz2uflmlsybvnq2irzn4ygyvu53oazyorednviid.onion/', '✔️', '', '', '']
[+] New row added! now writing the csv file:

[+] Want to add another website ? (y/n) n

Now that you added some websites to your unverified.csv file, you can view the csv file in www/participants/YOURINSTANCENAME.onion/unverified.csv:

[ Wonderland ] [ /dev/pts/33 ] [/srv/darknet-lantern]
→ cat www/participants/lantern.nowherejezfoltodf4jiyl6r56jnzintap5vyjlia7fkirfsnfizflqd.onion/unverified.csv | grep Tools

Instance,Category,Name,URL,Sensitive,Description,Status,Score

lantern.nowherejezfoltodf4jiyl6r56jnzintap5vyjlia7fkirfsnfizflqd.onion,Tools,Feather Wallet,http://featherdvtpi7ckdbkb2yxjfwx3oyvr3xjz3oo4rszylfzjdg6pbm3id.onion/,❌,Lightweight Monero Wallet,,
lantern.nowherejezfoltodf4jiyl6r56jnzintap5vyjlia7fkirfsnfizflqd.onion,Tools,Whonix Website,http://www.dds6qkxpwdeubwucdiaord2xgbbeyds25rbsgr73tbfpqpt4a6vjwsyd.onion/,❌,VM for general anonymous use,,
lantern.nowherejezfoltodf4jiyl6r56jnzintap5vyjlia7fkirfsnfizflqd.onion,Tools,Qubes OS Website,http://qubesosfasa4zl44o4tws22di6kepyzfeqv3tg4e3ztknltfxqrymdad.onion/,❌,OS based on Xen that focuses on compartmentalization and virtualization.,,

7. As you can see, the websites' uptime status and score is missing (in the last 2 columns in the csv file), therefore using scripts/uptimechecker.py we'll automatically fill those in:

[ Wonderland ] [ /dev/pts/33 ] [/srv/darknet-lantern]
→ python3 scripts/uptimechecker.py
[+] ONION UPTIME CHECKER
[+] Instance Name: lantern.nowherejezfoltodf4jiyl6r56jnzintap5vyjlia7fkirfsnfizflqd.onion True
[+] Reading the CSV File: /srv/darknet-lantern/www/participants/lantern.nowherejezfoltodf4jiyl6r56jnzintap5vyjlia7fkirfsnfizflqd.onion/verified.csv
                        Name                                                URL
0        Hackliberty Writers  http://lvgjoige2hl5qm5xcxhxuulyhdnq2wk3277eu34...
1          The Nihilism Blog  http://blog.nowherejezfoltodf4jiyl6r56jnzintap...
2               SimpleX Chat                              https://simplex.chat/
3          Hackliberty Forum  http://yw7nc56v4nsudvwewhmhhwltxpncedfuc43qbub...
4                      Dread  http://g66ol3eb5ujdckzqqfmjsbpdjufmjd5nsgdipvx...
5                      Pitch  http://pitchzzzoot5i4cpsblu2d5poifsyixo5r4litx...
6   Hackliberty main website  http://kj3wvs3wyfhm3uhhuqxlrhhcp6dneuau4mmvptl...
7            Hackliberty OTS                       https://ots.hackliberty.org/
8                  DNM Bible  http://biblemeowimkh3utujmhm6oh2oeb3ubjw2lpgeq...
9            Psychonaut Wiki  http://vvedndyt433kopnhv6vejxnut54y5752vpxshja...
10                  Tor Taxi  http://tortaxi2dev6xjwbaydqzla77rrnth7yn2oqzjf...
11            LibreTranslate  http://translate.nowherejezfoltodf4jiyl6r56jnz...
12                Safetwitch  http://safetwitch.nowherejezfoltodf4jiyl6r56jn...
13                Privatebin  http://bin.nowherejezfoltodf4jiyl6r56jnzintap5...
14                    Redlib  http://redlib.nowherejezfoltodf4jiyl6r56jnzint...
15                Lain Radio  http://radio.nowherejezfoltodf4jiyl6r56jnzinta...
16                    Gothub  http://gothub.nowherejezfoltodf4jiyl6r56jnzint...
17            Forgejo Datura  http://git.nowherejezfoltodf4jiyl6r56jnzintap5...
18                 XMRBazaar                             https://xmrbazaar.com/
19                Crypton sh  http://cryptonx6nsmspsnpicuihgmbbz3qvro4na35od...
20                   SMSPool                               https://smspool.net/
21               Silent Link  http://silentlnit5ryavvfz5vw7s4qg62jujd666lnc4...
22                       JMP                                  https://jmp.chat/
23                MullvadVPN  http://o54hon2e2vj6c7m3aqqu6uyece65by3vgoxxhlq...
24                  IncogNET  http://incoghostm2dytlqdiaj3lmtn7x2l5gb76jhabb...
25                   Cockbox  http://dwtqmjzvn2c6z2x462mmbd34ugjjrodowtul4jf...
26                   NiceVPS  https://nicevpsvzo5o6mtvvdiurhkemnv7335f74tjk4...
27                 Kyun Host  http://kyunnnckhnkl6oevonhwbltenwbgxwxf54mcpvm...
28              Servers Guru  http://srvguru7bjzzjba7xy2hnx2ju4k77qy4eum2h3t...
29                  Nowhere2  http://webring.nowhevi57f4lxxd6db43miewcsgtova...
30                   Nowhere  http://uptime.nowherejezfoltodf4jiyl6r56jnzint...
[+] Checking if each .onion link is reachable:
[+] Editing the uptime score
0
[+] http://lvgjoige2hl5qm5xcxhxuulyhdnq2wk3277eu34zpukxvacmvwva6vid.onion/read 200
http://lvgjoige2hl5qm5xcxhxuulyhdnq2wk3277eu34zpukxvacmvwva6vid.onion/read ✔️
[+] Editing the uptime score
1
[+] http://blog.nowherejezfoltodf4jiyl6r56jnzintap5vyjlia7fkirfsnfizflqd.onion/ 200
http://blog.nowherejezfoltodf4jiyl6r56jnzintap5vyjlia7fkirfsnfizflqd.onion/ ✔️
[+] Editing the uptime score
2

[...]

It may take a few minutes depending on the size of your list of links to check all of their uptimes, as the script has to connect through Tor for every website to tell if they are reachable or not.

8. make sure the cronjob for scripts/uptimechecker.py is running at least once every 3 hours so that the csv files are automatically kept up to date.

[ Wonderland ] [ /dev/pts/23 ] [/srv/darknet-lantern]
→ crontab -e

*/3 0 * * * python3 /srv/darknet-lantern/scripts/uptimechecker.py 

can I do the initial setup ?

As detailed above, this project aims to gain traction over time as more and more people join the webring, so that a decentralized, peer to peer, anonymous search engine can be formed accross multiple communities.

What is the default trust model ?

When spinning up your own Darknet Lantern instance, you are automatically creating your instance folder in www/participants/yourinstancehostname.onion/ and in there you'll find a copy of the blank verified.csv, unverified.csv, and webring-participants.csv files, along with the default banner.png file.

By default, as they are in the templates/ directory, the verified.csv and unverified.csv files are empty as you shouldn't trust random links that other people are trusting, you should verify them yourself one by one, to make sure that they aren't malicious.

That is why by default, when you first setup your own lantern instance, you are not listing anything, nor trusting any links, nor trusting any webring participants. This is a safety measure by default to avoid vector of abuse.

What is the lifecycle of a Lantern instance ?

At first, you have an empty Lantern, no links listed, no links trusted, no webring participants trusted either.

If you want to go your own way to explore the darknet yourself, you can start to list links yourself, into the unverified.csv file, and later on once you browse them enough to verify their authenticity, you may choose to trust them, to move them from unverified.csv to verified.csv

If you want to avoid doing the same work that other webring participants may have already done, you may choose to synchronize with their lists of links, and to add the ones that you didn't list yet into your own unverified.csv file. That way you'll be able to start exploring the Darknet starting from where others's progress.

After a while of adding new links and verifying them, you may choose to trust another webring participant's list of links, to automatically add their verified links into your own verified.csv file. However be warned that this is risky due to the fact that you may inadvertently trust malicious links.

What if there are malicious Lantern Instances ?

As stated above, it is possible that there will be malicious Lantern instances, meaning that you may have malicious peers that start to list some extremely illegal websites into their verified.csv file. This is why by default you are not trusting any Lantern webring participant, nor any links.

This is also the reason why there is a blacklist.csv file. That way, if there are any malicious keywords or links to be found, they will be immediately deleted. I for instance, refuse to list any porn sites and i encourage you to do the same, as those can link to some extremely illegal content, that you definitely don't want to get associated with.

Using the blacklist.csv file you can list blacklisted words or links that will automatically be removed if encountered by the python scripts.

When should I mark websites as sensitive ?

In order to avoid putting all eggs in one basket, i recommend marking websites that are related to Drugs (as this is the most popular sensitive darknet topic) as sensitive whenever you add them into your unverified.csv file.

There is also a sensitive.csv file in your instance directory to list those keywords, so that they get automatically marked as sensitive by the python scripts.

How can I add more links ?

Use the scripts/lantern.py to do the following:

How can I verify links ?

Use the scripts/lantern.py to do the following:

How can I edit the wordlists ?

Use the scripts/lantern.py to do the following:

OLD (for archival purposes)

1. run your own darknet-lantern instance as detailed below

2) As you keep on exploring the darknet, make sure you list the new onion links you find in the CSV files that are in www/links/, (i like to keep them separated in categories, so that they can be displayed in groups in the php pages afterward.)

<u>Option A:</u>

3) If you want to contribute your findings to the master repository that i'm running, just [ping me directly on simplex](https://simplex.chat/contact#/?v=2-5&smp=smp%3A%2F%2F1OwYGt-yqOfe2IyVHhxz3ohqo3aCCMjtB-8wn4X_aoY%3D%40smp11.simplex.im%2FsM96T2YMg4As7FiVoUVEttBgZQVPxWnb%23%2F%3Fv%3D1-2%26dh%3DMCowBQYDK2VuAyEAsKONXcyHAFEd79i13QZf8vDRUU5dVgQKQqUy_0S7ay4%253D%26srv%3D6ioorbm6i3yxmuoezrhjk6f6qgkc4syabh7m3so74xunb5nzr4pwgfqd.onion) so that i can create you a gitea account first 

4) Then create a pull request so that i can add your own links (AND the link to your list of links too in the index.php page) to the git repo directly. (**Please make sure you are keeping the sensitive websites in the same page, so the risk-based separation is kept (non-sensitive websites / sensitive websites))**

<u>Option B:</u>

3) If you want to go your own way just let us know that you're running your own list of onion links in the [Darknet Exploration SimpleX Chatroom](https://simplex.chat/contact#/?v=2-7&smp=smp%3A%2F%2FL5jrGV2L_Bb20Oj0aE4Gn-m5AHet9XdpYDotiqpcpGc%3D%40nowhere.moe%2FDkWtBNDdIPVjaap5trLwtmBBEUjyqxSZ%23%2F%3Fv%3D1-3%26dh%3DMCowBQYDK2VuAyEAcQLIsWEJ0xCv9sMSYVQV_sfMTkK_pisAtN55jgH08T8%253D%26srv%3Dt3srl5etd5y7nfvamlkdsjmm2okfu7db2jp4crxvf3tkwo42odme3kid.onion&data=%7B%22groupLinkId%22%3A%22iK4_U76rW6AdmZZfgVDDVQ%3D%3D%22%7D) so that we can add the link to your list of links in our own.


4) make sure you spread awareness of the darknet onion webring on darknet forums (there's [dread](http://g66ol3eb5ujdckzqqfmjsbpdjufmjd5nsgdipvxmsh7rckzlhywlzlqd.onion/) for example)


## How to run your own Darknet Onion Webring instance ?