updated

README.md

@@ -622,18 +622,162 @@ porn
 ```
 
 
-## TODO: How can I edit the sensitive.csv and blacklist.csv wordlists ?
-```sh
-```
+## How can I edit the sensitive.csv and blacklist.csv wordlists ?
+
+To edit the sensitive.csv if you want to add a word or a link to flag as sensitive, you can do the following by using scripts/lantern.py: 
 
-Use the scripts/lantern.py to do the following:
 ```sh
 
+Select Option? (0-11): 7
+7
+[+] Add/Remove Words/URLs in the sensitive list (ex: drug)
+[+] do you want to 1) add  or 2) remove Words/URLs? (type exit to exit) 1
+[+] which Sensitive word do you want to add? (write -1 to exit) Drugz
+[+] checking if the Word/URL is valid:
+False False True
+[+] Word/URL is valid, adding the word into the sensitive wordlist
+[+] NEWROW= ['Drugz']
+[+] New row added! now writing the csv file:
+[+] which Sensitive word do you want to add? (write -1 to exit) -1
+
+[ Datura ] [ /dev/pts/18 ] [/srv/darknet-lantern]
+→ cat www/participants/lantern.nowhevi57f4lxxd6db43miewcsgtovakbh6v5f52ci7csc2yjzy5rnid.onion/sensitive.csv
+sensitive-words
+Drugz
+Market
+market
+drug
+
+
+Select Option? (0-11): 7
+7
+[+] Add/Remove Words/URLs in the sensitive list (ex: drug)
+[+] do you want to 1) add  or 2) remove Words/URLs? (type exit to exit) 2
+  sensitive-words
+0           Drugz
+1          Market
+2          market
+3            drug
+which word do you want to remove? (index 0 to (max index) (write -1 to exit) 0
+[+] removing selected index:
+  sensitive-words
+1          Market
+2          market
+3            drug
+which word do you want to remove? (index 0 to (max index) (write -1 to exit) -1
+
+```
+And to do the same but with the blacklist.csv file you can do the following:
+```sh
+Select Option? (0-11): 8
+8
+[+] Add/Remove words in the blacklist list (ex: porn)
+[+] Do you want to 1) add  or 2) remove Words/URLs? (type exit to exit) 1
+[+] Which Sensitive word do you want to add? (write -1 to exit) pr0N
+[+] Checking if the Word/URL is valid:
+False False True
+[+] Word/URL is valid, adding the word into the blacklist
+[+] NEWROW= ['pr0N']
+[+] New row added! now writing the csv file:
+[+] Which Sensitive word do you want to add? (write -1 to exit) -1
+
+
+Select Option? (0-11): 8
+8
+[+] Add/Remove words in the blacklist list (ex: porn)
+[+] Do you want to 1) add  or 2) remove Words/URLs? (type exit to exit) 2
+  blacklisted-words
+0              pr0N
+1              pr0n
+2              pron
+3              teen
+4              porn
+which word do you want to remove? (index 0 to (max index) (write -1 to exit) 0
+[+] removing selected index:
+  blacklisted-words
+1              pr0n
+2              pron
+3              teen
+4              porn
+which word do you want to remove? (index 0 to (max index) (write -1 to exit) -1
+
+```
+# How do i delete duplicates from my verified.csv and unverified.csv file ?
+you may sometimes add the same link twice so to avoid manually having to find such link you can simply do the following using scripts/lantern.py:
+```sh
+
+Select Option? (0-11): 9
+9
+[+] 9) Cleaning up all duplicates in your own unverified + verified.csv (based on the url)
+/srv/darknet-lantern/www/participants/lantern.nowhevi57f4lxxd6db43miewcsgtovakbh6v5f52ci7csc2yjzy5rnid.onion/verified.csv
+REMOVING DUPLICATES IN /srv/darknet-lantern/www/participants/lantern.nowhevi57f4lxxd6db43miewcsgtovakbh6v5f52ci7csc2yjzy5rnid.onion/verified.csv
+                                                  URL
+0   http://lvgjoige2hl5qm5xcxhxuulyhdnq2wk3277eu34...
+1   http://blog.nowherejezfoltodf4jiyl6r56jnzintap...
+2                               https://simplex.chat/
+3   http://yw7nc56v4nsudvwewhmhhwltxpncedfuc43qbub...
+4   http://pitchzzzoot5i4cpsblu2d5poifsyixo5r4litx...
+5   http://kj3wvs3wyfhm3uhhuqxlrhhcp6dneuau4mmvptl...
+6                        https://ots.hackliberty.org/
+7   http://biblemeowimkh3utujmhm6oh2oeb3ubjw2lpgeq...
+8   http://vvedndyt433kopnhv6vejxnut54y5752vpxshja...
+9   http://tortaxi2dev6xjwbaydqzla77rrnth7yn2oqzjf...
+10  http://translate.nowherejezfoltodf4jiyl6r56jnz...
+11  http://safetwitch.nowherejezfoltodf4jiyl6r56jn...
+12  http://bin.nowherejezfoltodf4jiyl6r56jnzintap5...
+13  http://redlib.nowherejezfoltodf4jiyl6r56jnzint...
+14  http://radio.nowherejezfoltodf4jiyl6r56jnzinta...
+15  http://gothub.nowherejezfoltodf4jiyl6r56jnzint...
+16  http://git.nowherejezfoltodf4jiyl6r56jnzintap5...
+17  http://cryptonx6nsmspsnpicuihgmbbz3qvro4na35od...
+18                               https://smspool.net/
+19  http://silentlnit5ryavvfz5vw7s4qg62jujd666lnc4...
+20                                  https://jmp.chat/
+21  http://o54hon2e2vj6c7m3aqqu6uyece65by3vgoxxhlq...
+22  http://dwtqmjzvn2c6z2x462mmbd34ugjjrodowtul4jf...
+23  https://nicevpsvzo5o6mtvvdiurhkemnv7335f74tjk4...
+24  http://kyunnnckhnkl6oevonhwbltenwbgxwxf54mcpvm...
+25  http://srvguru7bjzzjba7xy2hnx2ju4k77qy4eum2h3t...
+26  http://webring.nowhevi57f4lxxd6db43miewcsgtova...
+27  http://uptime.nowherejezfoltodf4jiyl6r56jnzint...
+/srv/darknet-lantern/www/participants/lantern.nowhevi57f4lxxd6db43miewcsgtovakbh6v5f52ci7csc2yjzy5rnid.onion/unverified.csv
+REMOVING DUPLICATES IN /srv/darknet-lantern/www/participants/lantern.nowhevi57f4lxxd6db43miewcsgtovakbh6v5f52ci7csc2yjzy5rnid.onion/unverified.csv
+                                                  URL
+0                            https://houdiniswap.com/
+1                                     https://crp.is/
+2                                https://swapzone.io/
+3                              https://tradeogre.com/
+4   http://wizardswgtu2ovor7r2esg3cxdpt7tv4nrugi32...
+..                                                ...
+74                             https://safing.io/spn/
+75                               https://mynymbox.io/
+76                         https://orangewebsite.com/
+77  http://ax3zoslh2ujdq3joyibdn657mhjfxjm637vxiix...
+78  http://spore64i5sofqlfz5gq2ju4msgzojjwifls7rok...
+
+[78 rows x 1 columns]
+
 ```
 
+
+## How do i perform sanity checks on all links from all webring participants ? 
+If you want to make sure every link of every webring participant is properly flagged as "sensitive" and/or blacklisted, you can do the following using scripts/lantern.py:
+
 ```sh
+Select Option? (0-11): 10
+10
+[+] 10) perform sanity checks on all csv files (to mark them as sensitive / or remove the ones that are blacklisted)
+Participant: lantern.nowherejezfoltodf4jiyl6r56jnzintap5vyjlia7fkirfsnfizflqd.onion
+/srv/darknet-lantern/www/participants/lantern.nowherejezfoltodf4jiyl6r56jnzintap5vyjlia7fkirfsnfizflqd.onion/verified.csv
+['lantern.nowhevi57f4lxxd6db43miewcsgtovakbh6v5f52ci7csc2yjzy5rnid.onion', 'Blogs', 'Hackliberty Writers', 'http://lvgjoige2hl5qm5xcxhxuulyhdnq2wk3277eu34zpukxvacmvwva6vid.onion/read', '❌', nan, '✔️', 100.0]
+
+[...]
+
+['lantern.nowhevi57f4lxxd6db43miewcsgtovakbh6v5f52ci7csc2yjzy5rnid.onion', 'Forums', 'Dread', 'http://g66ol3eb5ujdckzqqfmjsbpdjufmjd5nsgdipvxmsh7rckzlhywlzlqd.onion/', '✔️', 'Dread, probably the biggest!ù*$&=)"à&è'& darknet forum out there, definitely worth checking out, the main topic there is Drugs', '❌', 59.0]
+Marking row 4 for deletion, as it has invalid inputs
+
 ```
-
+Here for example it just found an entry with invalid inputs, hence it has been marked for deletion