If you explore too much, you waste your time (among other resources.) You will probably exhaust your resources before you learn anything meaningful.
Let's say your goal is to learn as much about Star Wars as possible within a library. If you fully explore, you just pick books at random.
Exploitation might look something like "pick most of your books within the Sci-Fi section""pick most of your books within the Sci-Fi section", or "choose books with lightsabers on the cover""choose books with lightsabers on the cover" or "books with 'Star Wars' in the title""books with 'Star Wars' in the title" because that's where you have found relevant information in the past.