Improving Crawler – Intro to Computer Science

Improving Crawler – Intro to Computer Science


If we want more confidence, we could also look at the documentation and the challenges. If we don’t know what we’re looking for, well the documentation won’t tell us We can see that we have union that returns a union of the sets. What we don’t see yet is update and now, given that we found update and guess what it did, we can see this description, not the most clear description to know for sure that it does what we want. We could do some more searches. Let’s see what we got when we search for Python and update. Well, we got some discussion here, we’re looking at the dictionary update. Let’s make sure we are looking for the set update and we get some more documentation. This is going back to what we’re looking for probably not the most useful thing. And now searching for update, we see this documentation, the same that we saw. It is updating the set adding elements from all others. So, I think it is easier to understand what something does by experimentation than by reading the documentation. You should be open to using both approaches and combining them with looking at example code and trying things own your own. So, for the final quiz of this mini unit, your goal is to modify crawl_web that was part of the web crawler that we developed in CS101 to use set objects for tocrawl and crawled instead of using list. You should make your search engine simpler and remove the need for the separate union function.

Leave a Reply

Your email address will not be published. Required fields are marked *