Docs not building on hackage

[axman6]

Looks like the build of the docs on Hackage is failing at the moment: https://hackage.haskell.org/package/dataframe-0.2.0.2/reports/2 It'd be great to get the issues fixed to make the docs available. Alternatively, you can upload the docs yourself, https://hackage.haskell.org/upload had the details on generating locally and uploading at the end.

[mchav]

Yeah. This was an issue with the snappy bindings. I think I'd have had to do a custom install script that also installs libsnappy-dev. Hence why I ended up rewriting snappy for an easier installation story. The latest Hackage version should work though: 0.3.0.1

[axman6]

Oh my mistake! I just followed the hackage link from the readme, which links to that version for some reason! Thanks for pointing that out.

[axman6]

By the way, your post on Reddit about your Snappy implementation got me really interested in writing something faster (using both Builder and concatenating bytestrings in each recursive call gives you basically the worst of both worlds, so doing something in ST which can lazily produce chunks feels like the way to go to me). I'll have to find some time to see if I can make you something faster.

Also I'd be interested hearing your thoughts on haskell/statistics#220, one of the usecases I had in mind for this work was dataframe-esque libraries, where being tied to only Vector v => v Double might lead to performance issues. I wanted to expose something that you could incorporate accessor functions with, so columns could have more complex types:

import Statistics.Sample as F
import Statistics.Sample.Fold as F

-- Checks that vectors contain `a` omitted
meanOf :: Columnable a => (a -> Double) -> Column -> Maybe Double
meanOf f (BoxedColumn xs) = F.foldf (premap f mean) xs
meanOf f (UnboxedColumn xs) = F.foldf (premap f mean) xs
meanOf f (OptionalColumn xs) = F.foldf (F.prefilter isJust (F.premap (f . fromJust) mean)) xs

And in general being able to apply Fold a b over any column would be quite useful

aggregateColumnFold :: Fold a b -> Column -> Maybe b

aggregateFold :: Text -> Fold a b -> DataFrame -> DataFrame -> Maybe b


-- How to handle optional columns isn't clear, but probably only include any result if all data is there
aggregateColumns  :: (Text, Text)       -> Fold (a,b) c   -> DataFrame -> Maybe c
aggregateColumns3 :: (Text, Text, Text) -> Fold (a,b,c) d -> DataFrame -> Maybe d

-- Generate a column as from a fold scanned over the data
scanColumn :: Columnable b => Fold a b -> Column -> Maybe Column

[mchav]

I have a local branch that hadn't profiled yet cause I also realized the statistics library is a quarter of the compilation time. Since I only used a few functions I thought to just reimplement them.

Let me open that investigation back up and profile it.