Welcome to my first Haskell post of 2015, where I hope to show you how to make the Haskell REPL (GHCi) work even better for you!
Note - this post covers much of the same ground as this one. It adds:
This guide features code samples, file paths, and timings towards integrating GHCi with Hoogle, and what that gives you. In bullet form, the guide proceeds as follows:
Let’s begin!
Hoogle is both a web search engine and a command line utility for searching for:
foldl, relativizeUrls, async(a -> Bool) -> [a] -> [a], a -> aFold, Textasync, containersHere’s some examples in action:
$ hoogle --count=5 'foldl'
Prelude foldl :: (a -> b -> a) -> a -> [b] -> a
Data.List foldl :: (a -> b -> a) -> a -> [b] -> a
Data.Foldable foldl :: Foldable t => (a -> b -> a) -> a -> t b -> a
Data.ByteString.Char8 foldl :: (a -> Char -> a) -> a -> ByteString -> a
Data.ByteString.Lazy.Char8 foldl :: (a -> Char -> a) -> a -> ByteString -> a
$
$ hoogle --count=5 'con map'
Data.ByteString.Char8 concatMap :: (Char -> ByteString) -> ByteString -> ByteString
Data.ByteString.Lazy.Char8 concatMap :: (Char -> ByteString) -> ByteString -> ByteString
Data.Text.Internal.Fusion.Common concatMap :: (Char -> Stream Char) -> Stream Char -> Stream Char
Data.Text concatMap :: (Char -> Text) -> Text -> Text
Data.Text.Lazy concatMap :: (Char -> Text) -> Text -> Text
$
$ hoogle --count=5 '(a -> Bool) -> [a] -> [a]'
Prelude dropWhile :: (a -> Bool) -> [a] -> [a]
Data.List dropWhile :: (a -> Bool) -> [a] -> [a]
Prelude filter :: (a -> Bool) -> [a] -> [a]
Data.List filter :: (a -> Bool) -> [a] -> [a]
Prelude takeWhile :: (a -> Bool) -> [a] -> [a]
$
$ hoogle --count=5 'Maybe'
Prelude data Maybe a
Data.Maybe data Maybe a
Data.Maybe module Data.Maybe
Control.Monad.Trans.Maybe module Control.Monad.Trans.Maybe
Control.Monad.Trans.Maybe MaybeT :: m (Maybe a) -> MaybeT m aHow fast is it?
$ time hoogle --count=5 '(a -> Bool) -> [a] -> [a]'
Prelude dropWhile :: (a -> Bool) -> [a] -> [a]
Data.List dropWhile :: (a -> Bool) -> [a] -> [a]
Prelude filter :: (a -> Bool) -> [a] -> [a]
Data.List filter :: (a -> Bool) -> [a] -> [a]
Prelude takeWhile :: (a -> Bool) -> [a] -> [a]
real 0m0.136s
user 0m0.112s
sys 0m0.024sAbout ~100 milliseconds for a polymorphic query. That’s fast enough to be interactive! (note: YMMV. Hoogle accesses local storage, finds matches, computes ranks, and displays matches by best-score-first. I have an HDD, so you’d get even better results with an SSD, most likely.)
What if it can’t find a match?
We can also mimic the behavior of Python’s help(x), Ruby’s (via Pry) show-doc x, or Clojure’s (doc x) via Hoogle. Here’s a few examples:
$ hoogle --info 'Maybe'
Prelude data Maybe a
The Maybe type encapsulates an optional value. A value of type Maybe a
either contains a value of type a (represented as Just a), or it is
empty (represented as Nothing). Using Maybe is a good way to deal with
errors or exceptional cases without resorting to drastic measures such
as error.
The Maybe type is also a monad. It is a simple kind of error monad,
error monad can be built using the Data.Either.Either type.
From package base
data Maybe a
$
$ hoogle --info 'map'
Prelude map :: (a -> b) -> [a] -> [b]
map f xs is the list obtained by applying f to each element of xs,
i.e.,
> map f [x1, x2, ..., xn] == [f x1, f x2, ..., f xn]
> map f [x1, x2, ...] == [f x1, f x2, ...]
From package base
map :: (a -> b) -> [a] -> [b]
$
$ hoogle --info '(<$>)'
Data.Functor (<$>) :: Functor f => (a -> b) -> f a -> f b
An infix synonym for fmap.
From package base
(<$>) :: Functor f => (a -> b) -> f a -> f bWith hoogle and hoogle --info, we can go far! Let’s get it installed.
The first step to installing Hoogle is installing Haskell and cabal on your host. I defer to this guide to walk you through the details, whether you’re on Windows, Mac OS X, or a Linux distribution.
With that set, now we’ll install hoogle in a sandbox:
$ mkdir <some_place>/hoogle
$ cd <some_place>/hoogle
$ cabal sandbox init
Writing a default package environment file to
<some_place>/hoogle/cabal.sandbox.config
Creating a new sandbox at <some_place>/hoogle/.cabal-sandbox
$ cabal update
$ cabal install hoogle -j # Long step: compiles lots of packages
$ cd .cabal-sandbox/bin
$ pwd
<some_place>/hoogle/.cabal-sandbox/binTake the path generated by the last command (pwd, or echo %cd% if on Windows) and add that to your PATH. This will make it possible to use hoogle from any path on your system.
Take note that your Hoogle databases are also stored near this path. For my particular sandbox location:
We’ll cover working with Hoogle databases a bit more in the coming section.
Hoogle isn’t very helpful without local data. It’ll even tell you this if you try to run it without grabbing the required data.
$ hoogle map
Could not find some databases: default
Searching in:
.
/home/allele/haskell/hoogle/.cabal-sandbox/share/x86_64-linux-ghc-7.8.3/hoogle-4.2.36/databases
There are no available databases, generate them with: hoogle dataWhat is the default database? In the previous section, we talked about a Hoogle databases path. In this path, if you’ve generated data, you’ll find a default.hoo. Let’s try another failing search before proceeding:
$ hoogle map +asyn +fiddling
Could not find some databases: asynca fiddling
Searching in:
.
/home/allele/haskell/hoogle/.cabal-sandbox/share/x86_64-linux-ghc-7.8.3/hoogle-4.2.36/databases
Either the package does not exist or has not been generated.This tells us a bit more about how Hoogle works. Given a set of databases to work with indicated by +<name> arguments, it’ll load those and perform its search. If no +<name> arguments are found, Hoogle works as if +default has been specified.
With that introspection out of the way, let’s generate some data!
If this gives you trouble on Windows, check out this Stack Overflow question.
This will data for a few packages - enough information to match up with the libraries the Haskell Platform installs. This includes things like:
If you need more packages, there’s two approaches. You can: 1) generate Hoogle DBs for all the packages on Hackage using a single command, or 2) fine-grained Hoogle DB generation with a series of commands.
I’ll only cover 1) in this post. I’ll refer you to Eric Rasmussen’s guide for fine-grained control. (Search: “How to Hoogle: the hard way”).
To generate data for all the packages:
This will take quite awhile. It took over an hour on my machine, and more than 7 GB of RAM. It might fail with an out-of-memory error. It’s safe to restart the process and it’ll pick up where it left off. It’s handy to have a snapshot of the Hackage ecosystem on your machine, but it certainly takes a long time to generate and isn’t resource-friendly.
What if you want to generate data for projects you’re working on locally? Try this:
$ cd <project>
$ cabal configure; # sandbox, install any dependencies needed
$ cabal haddock --hoogle
$ hoogle convert dist/doc/html/<project>/<project>.txt
$ cp dist/doc/html/<project>/<project>.hoo <HOOGLE_DB_PATH><HOOGLE_DB_PATH> is the path I pointed out at the end of the previous section. By copying the generated <project>.hoo file over, it’s information is now available to Hoogle when given +<project>. Here’s an example involving one of my projects:
$ hoogle --info parse +brainfuck-tut
Language.Brainfuck.Parse parse :: String -> [Term]
A total function over the BF syntax.
From package brainfuck-tut
parse :: String -> [Term]It’s not a convenient process. It’s eased by putting together a script to do the repetitive bits for you, but this would be something that’d be lovely to have automated as part of cabal haddock --hoogle with or without a sandbox.
The last step in this guide is bringing together GHCi and Hoogle. The end result will be that we’ll have registered special commands in GHCi so we can interact with Hoogle without leaving the REPL. Here’s what it looks like:
> :hoogle head
Searching for: head
Prelude head :: [a] -> a
Data.List head :: [a] -> a
Data.ByteString.Char8 head :: ByteString -> Char
Data.ByteString.Lazy.Char8 head :: ByteString -> Char
Data.ByteString.Lazy head :: ByteString -> Word8
Data.ByteString head :: ByteString -> Word8
Data.Text.Internal.Fusion.Common head :: Stream Char -> Char
Data.Text head :: Text -> Char
Data.Text.Lazy head :: Text -> Char
Text.Html header :: Html -> Html
>
> :doc head
Prelude head :: [a] -> a
Extract the first element of a list, which must be non-empty.
From package base
head :: [a] -> a
>
> :doc Data.Text.Lazy.head
Data.Text head :: Text -> Char
O(1) Returns the first character of a Text, which must be non-empty. Subject to fusion.
From package text
head :: Text -> CharTo enable this, it takes modifying our .ghci file. On Unix-like systems, the path should be ~/.ghci: at the base of our $HOME directory. On Windows, refer to this guide.
Add the following lines to .ghci:
:def hoogle \x -> return $ ":!hoogle -c -n 10 \"" ++ x ++ "\""
:def doc \x -> return $ ":!hoogle --info \"" ++ x ++ "\""The -n 10 is the number of results we want returned when we search. Modify that to suite your preferences. -c adds a little bit of color support to hoogle queries. On my terminal, the terms matched are underlined and bold.
The :def command for GHCi, more generally, takes a function \x -> ... that captures arguments passed as String and passes them to GHCi to interpret. The !hoogle portion says we’re invoking a shell process named hoogle and passing it the remaining characters.
This is good to know if you’d like to extend GHCi further with your own short-hand commands!
It’s kind of silly that it took me this long to realize that Hoogle had the ability to provide documentation at the REPL. Even if it isn’t (Haskell’s a pretty big space to explore!), I feel silly.
I’ve had the set up above for most of 2014. I relied heavily on :hoogle <name | type> during the early parts of my Haskell education, but never realized I could also look up docs!
Instead of using :doc, I’d load up the browser, search for the package I was working with on Hackage, and look over the documentation as I ran into questions. I did this a lot with attoparsec, which is a well-documented and very capable parser.
All this time, I could’ve just been working from the REPL. Hah…
We’re coming close to the end of this post. In writing it, I ran into a few issues. Let’s see how these match up with recorded issues on Hoogle’s github repository.
hoogle data isn’t incremental: known
hoogle --d <dir1> --d <dir2> helps.cabal sandbox awarenessHoogle’s a wonderful tool. It makes working with Haskell more of a pleasure, and it certainly helps when learning to navigate the Haskell ecosystem.
I hope this guide has helped!
For further reading, check out some of the links below.