Showing posts with label haskell. Show all posts
Showing posts with label haskell. Show all posts

Thursday, July 14, 2022

Haskell and data analytics

Here's a question:

Can Haskell be used to analyze data published by, say @KingdomInformed, on, say: @DefiKingdoms heroes for sale?

Answer: welp, let's find out.

We'll answer the above question by posing then answering smaller, simpler questions.

Question 1: can I look at only Serendale heroes?

Answer: yes! https://dfkanalytics.vercel.app/hero select Serendale Realm on filters. 


Question 2: can I download the stats for Serendale?

Answer: yes! Yes, I can, via the 'export'-link. 


Parsing DFK Heroes-for-sale Data


Question 3: can I parse this file?

Answer: YES! ... but this is divided into simpler questions.

Question 3.a: What are the relevant hero data from the CSV file?

Answer: These are, ...


... which we capture in this Haskell data-type.


Question 3.b: Do I want per-line error reporting?
Answer: yeup.


Question 3.c: Does that mean I need to create a language around the DFK hero knowledge-base?
Answer: But of course!


Okay, before we get into the guts of the parser: can we parse in the DFK hero sales data?

Yes, yes we can. 


What do heroes sale data look like when parsed?

Answer: like this. 


Alrighty, then. What does the parser look like?

Answer: At a high level, we ingest the CSV file, scan each line, then parse data from each line into the Hero-data-structure.


What does 'parsing at a low level' look like?

Answer: It looks like this. We match each column of CSV to each datum in the Hero-structure, capturing if we can't at each datum as errors which we report out.

Fortunately, there were no errors, this parse. 😅🎉


Constructing the individual parsers was a reductive exercise, fortunately. Why?

1. First, I constructed a Language of Parsing DFK heroes. 

Once you shape a language of the problem, the solution simply falls out.

2. and I constructed the generic `parseT` higher-order function.


Distribution Analyses

That's great! I can parse DFK data!

Who cares? (I do) 🥺

That's a valid question. Data are to be analyzed.

So, what analyses can we do?

1. We can compute distributions along any dimension of the hero-vectors.

Say: "What are the distribution of levels of heroes for sale?"


Okay, but all that work for just heroes for sale-by-level?

Nupe.

We can do heroes-for-sale by x, where x is any dimension of the Hero-vector.

Let's say, heroes-by-generation? How do we code that?

Easy!

> count gen hs

And we get:


How about "Heroes for sale-by-(primary)-class"?

Answer: np

> count class1 hs

Once we've parsed the data into our SVM, we can query any dimension's distribution simply. 


"What dark majick't is this `count`-function, geophf?" you demand vociferously.

Answer: um, ... it's just a distribution function. 


"So, ya gon' be a d!ck? Or are you gon' share your source code, huh, geophfrmeisterG?"

Me: "...yes?" 🤓😎 

Postlude and teaser


These are simple analyses. Useful? Sure!

But!

How about analyses considering several, or all, vectors?

How about the question: "Given 'a priori'-data, how much can I sell my hero, x, for?"

Would you like an answer for that question?

I would.

Tune in for bayesian analyses.

CrystalVale Hero Data


p.s.: "How many changes did I have to make to my system to parse DFK CrystalVale heroes-for-sale data?"

Answer: none. 



Wednesday, March 30, 2016

Standing up Haskell (GHC) on an AWS EC-2

Steps for standing up dev tools, and Haskell, on AWS EC-2

So, you’ve configured your EC-2 with the standard AWS-Linux image, and now you want to compile your sources and have running executables to demo, including in not only Java and Perl but possibly C, C++ and, for my prototypes and graph-systems, Haskell. How to do this?

Unfortunately, the AMI for Haskell is rather unforthcoming, as it seems to compile the GHC from sources, an endeavor that takes hours of AWS-time. Unnecessary. So, I’ve cut out those middle steps and have a Haskell-image on my S3 that can be downloaded and used once the dev-tools are in place.

Let’s do this.

1. Get the ghc image from my public-S3-bucket:
$ wget https://s3-us-west-2.amazonaws.com/haskell-7-10-2/haskell-platform-7.10.2-a-unknown-linux-deb7.tar.gz

2. install the dev-tools from Amazon (this takes a bit)

$ sudo yum groupinstall “Development Tools”

3. Now you need to provide a soft link to libgmp.so:

$ cd /usr/lib64
$ sudo ln -s libgmp.so.3 libgmp.so
$ sudo ln -s libgmp.so.3 libgmp.so.10
$ cd

Yes, you need both of those soft links.

4. Once you have all that done, unzip ghc:

$ sudo tar xzf haskell-platform-7.10.2-a-unknown-linux-deb7.tar.gz

5. And now ghci will work for you:

$ ghci
Prelude> 3 + 4
~> 7

YAY! Do a happy dance!

Thursday, December 17, 2015

Haskell Trading Analytics Libraries

Logical Graphs LLC has a set of Haskell libraries based around technical indicators located at our trading analytics page.