add llm web search section

opsec/openwebuilocalllms/index.html

@@ -1263,18 +1263,141 @@ Only output the translation, nothing else.</code></pre>
 <div class="row">
 <div class="col-lg-8 col-lg-offset-2">
 <h2 id="troubleshooting"><b>Troubleshooting</b></h2>
-<p>If you encounter issues with hardware acceleration on ollama, check:</p>
+<p>If you encounter issues with hardware acceleration on ollama, check:
 <ul>
     <li>Ensure the NVIDIA driver is correctly installed and accessible within Docker.</li>
     <li>Verify GPU resources are allocated by running <b>docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi</b>.</li>
     <li>Check logs with <b>docker compose logs -f</b> for any error messages.</li>
 </ul>
+</p>
             </div>
         </div><!-- /row -->
     </div> <!-- /container -->
 </div><!-- /white -->
 
 <div id="anon1">
+  <div class="container">
+    <div class="row">
+      <div class="col-lg-8 col-lg-offset-2">
+<h2 id="llm-web-search"><b>[BONUS] LLM Web Search</b></h2>
+
+<p>
+LLMs typically have pretty distant knowledge cutoff. Meaning if you ask about recent developments in some rapidly changing technology, they typically won't be able to answer you directly.<br>
+The models presented here typically have the knowledge up to late 2023/early 2024 since they were trained somewhat around this time.
+</p>
+
+<p>
+With Open WebUI, models can search the web for up to date information about recent topics.<br>
+The model is first asked to come up with search queries about the question being asked. The queries are sent to some traditional search engine (like duckduckgo, bing, google or a searxng instance). A webdriver working on the backend visits each result and aggregates knowledge in vector database which model then uses to enhance its response.<br>
+In this section, we'll configure this feature to work entirely over Tor maintaining server side anonymity.</p>
+
+<p>
+Here's the output from Gemma 3 without search capability:
+</p>
+
+<img src="41.png" class="imgRz"><br><br>
+<img src="43.png" class="imgRz">
+
+<p><br>
+And here are the same questions with search using duckduckgo over Tor:
+</p>
+
+<img src="40.png" class="imgRz"><br><br>
+<img src="42.png" class="imgRz">
+
+<p><br>
+To start, we need to know the IP address of the host on the <b>docker0</b> interface.
+<pre><code class="nim">oxeo@andromeda:~$ ip a show dev docker0
+3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
+    link/ether 3a:1c:1f:86:47:f0 brd ff:ff:ff:ff:ff:ff
+    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
+       valid_lft forever preferred_lft forever</code></pre>
+In my case it's <b>172.17.0.1</b>.
+</p>
+
+<p>
+Now, we'll make Tor listen on this interface with <b>HTTP CONNECT</b> proxy (since Open WebUI search feature doesn't support socks5). Add at the top of <b>/etc/tor/torrc</b> file the following line:
+<pre><code class="nim">HTTPTunnelPort 172.17.0.1:9080</code></pre>
+Remember to replace the IP with the one you got from previous step.
+</p>
+
+<p>
+Restart Tor and check if it listens on desired interface:
+<pre><code class="nim">oxeo@andromeda:~$ sudo systemctl restart tor
+oxeo@andromeda:~$ sudo ss -tulp
+Netid  State    Recv-Q   Send-Q     Local Address:Port       Peer Address:Port  Process                                                                         
+udp    UNCONN   0        0                0.0.0.0:bootpc          0.0.0.0:*      users:(("dhclient",pid=490,fd=7))                                              
+tcp    LISTEN   0        4096           127.0.0.1:9050            0.0.0.0:*      users:(("tor",pid=1572,fd=6))                                                  
+tcp    LISTEN   0        4096           127.0.0.1:3000            0.0.0.0:*      users:(("docker-proxy",pid=13793,fd=7))                                        
+tcp    LISTEN   0        128              0.0.0.0:ssh             0.0.0.0:*      users:(("sshd",pid=522,fd=3))                                                  
+tcp    LISTEN   0        4096          172.17.0.1:9080            0.0.0.0:*      users:(("tor",pid=1572,fd=7))                                                  
+tcp    LISTEN   0        4096           127.0.0.1:11434           0.0.0.0:*      users:(("docker-proxy",pid=13708,fd=7))                                        
+tcp    LISTEN   0        128                 [::]:ssh                [::]:*      users:(("sshd",pid=522,fd=4))</code></pre>
+</p>
+
+<p>
+We also need to adjust the <b>~/openwebui-stack/docker-compose.yml</b> file.<br>
+Add 3 <a href="https://docs.openwebui.com/getting-started/env-configuration/#proxy-settings">environment variables</a> telling Open WebUI to use certain proxy for HTTP and HTTPS connections. The open-webui container configuration should now look like this:
+<pre><code class="nim">  open-webui:
+    image: ghcr.io/open-webui/open-webui:main
+    container_name: open-webui
+    volumes:
+      - open-webui:/app/backend/data
+    depends_on:
+      - ollama
+    ports:
+      - 127.0.0.1:3000:8080  # Remove "127.0.0.1:" to access from LAN
+    environment:
+      - 'OLLAMA_BASE_URL=http://ollama:11434'
+      - 'WEBUI_SECRET_KEY='
+      - 'HTTP_PROXY=http://host.docker.internal:9080'
+      - 'HTTPS_PROXY=http://host.docker.internal:9080'
+      - 'NO_PROXY=ollama'
+    extra_hosts:
+      - host.docker.internal:host-gateway
+    restart: unless-stopped</code></pre>
+The <b>host.docker.internal</b> domain is resolved from within the container to the address of host. This allows open-webui container to access HTTP CONNECT proxy exposed by Tor daemon. 
+</p>
+
+<p>
+Once that's done, we can restart the container and go to Open WebUI GUI administrator settings once again.
+<pre><code class="nim">oxeo@andromeda:~$ docker compose down; docker compose up -d</code></pre>
+</p>
+<img src="44.png" class="imgRz">
+
+<p><br>
+This time, click on the <b>Web Search</b> tab.
+</p>
+<img src="45.png" class="imgRz">
+
+<p><br>
+Here, enable the feature and select <b>duckduckgo</b> as a search engine. It's also possible to use searxng instance but it has to have JSON output enabled (which is not the default).<br>
+You should enable <b>Trust Proxy Environment</b> so that every search query and visited website will be proxied through the Tor proxy we set up before.
+</p>
+<img src="46.png" class="imgRz">
+
+<p><br>
+And that's it!<br>
+Now go back to the chat interface and click on <b>Web Search</b> toggle, then send your question.
+</p>
+<img src="47.png" class="imgRz">
+
+<p><br>
+If search engine and browser backend are working, you should see search queries model came up with.
+</p>
+<img src="48.png" class="imgRz">
+
+<p><br>
+After model gives you an answer, you're able to see the sources it used to gain knowledge.
+</p>
+<img src="49.png" class="imgRz">
+
+        </div>
+      </div><!-- /row -->
+  </div> <!-- /container -->
+</div><!-- /white -->
+
+<div id="anon2">
     <div class="container">
         <div class="row">
             <div class="col-lg-8 col-lg-offset-2">