add speed improvements
This commit is contained in:
		@@ -17,6 +17,8 @@ This ment we stored 1000 pages, 142,997 urls, and 1,425,798 links between the tw
 | 
				
			|||||||
 | 
					
 | 
				
			||||||
3/20/25: Took 5min to crawl 1000 pages
 | 
					3/20/25: Took 5min to crawl 1000 pages
 | 
				
			||||||
 | 
					
 | 
				
			||||||
 | 
					3/21/25: Took 3min to crawl 1000 pages
 | 
				
			||||||
 | 
					
 | 
				
			||||||
# About
 | 
					# About
 | 
				
			||||||
 | 
					
 | 
				
			||||||

 | 
					
 | 
				
			||||||
 
 | 
				
			|||||||
		Reference in New Issue
	
	Block a user