Integrating peroxide

[Mec-iS]

@Axect is a maintainer at peroxide, a nice linear algebra library. We can consider integrating some of its functionalities.

• What are the principles that inspired your implementation?
• what parts are more interesting and how did you approach the design decisions in terms of Rust traits and constructs?
• which strong points you would like to provide with your library?

In particular also:

• How did you integrate parquet and NetCDF files? are you considering using HDFS?
• what are the challenges of implementing gradient descent and gradient boost?

[Axect]

I'm very happy to collaborate with such a great library as smartcore. Thank you @Mec-iS !

Let me try to answer your first three questions.

• The overriding principle of Peroxide is that it should be easy even for people who have little experience with Rust. We also want to reduce external dependencies outside the Rust ecosystem as much as possible. Of course, when external dependencies such as OpenBLAS or netcdf are required, they are provided as features so that only those who want to use them.

• In terms of trait, it was designed to compatible with mathematics but practicality was emphasized more. For example,

  Trait	Description	Implemented types or structs
  Vector	If the object implements Vector then it can be considered as elements of the vector space	f64, Vec, Matrix, Series
  Normed	Represents to normed vector space	Vector
  InnerProduct	Represents to inner product vector space	Normed
  LinearOp	the linear map between two vector spaces	Vec, Matrix

  However I thought that being too mathematically rigorous might cause a barrier to entry, so each method was designed to be used easily. In addition, there are some useful traits too.

  Trait	Description	Implemented types or structs
  FPVec	Functional programming tools (frequently used in Haskell, Scala etc.)	Vec
  FPMat	Functional programming tools for Matrix	Matrix
  MatrixPtr	Low level programming tools for Matrix	Matrix
  Oxide, RedoxCommon	Help to operate vectors without transferring ownership	Vec

• The strengths of Peroxide in my opinion are as follows.

  • Ease of use (If you want to make matrix, then you can use R-like matrix constructor or Matlab-like or Python like either.)
  • Linear algebra and many numerical computations can all be used in one crate. Actually, I use peroxide to conduct cosmology, astrophysics, and particle physics - related research, and most of the calculations of researches can be solved with peroxide.
  • Customize option : In peroxide, there are two options - fuga and prelude. If you want to compute something with a great detail, then use fuga. Or just want to compute something simply, then use prelude.
  • features : There are some great features in peroxide.
    • O3 : For OpenBLAS
    • csv : Read & write DataFrame with csv
    • nc : Read & write DataFrame with netcdf
    • parquet : Read & write DataFrame with parquet

[Axect]

Now, I will answer the next two questions you asked.

First, I created DataFrame struct and Series struct, inspired by pandas, a popular dataframe library used in Python. Next, I wanted to make it simple to read an write to multiple file formats. So, I choose csv, NetCDF and parquet.

The implementation itself is not particularly special, there are already a great Rust libraries georust/netcdf and arrow2, so I implemented only the frontend to use it easily in the peroxide's DataFrame using these libraries as backends.

Of course, I also considered implementing HDFS, but Rust's excellent hdf5 library hdf5-rust requires ndarray, which overlaps a lot with peroxide, as default, so I didn't use that. Fortunately, the hdf5-rust team provides hdf5-sys, which is a binding for the HDF5 C library, so I want to using it, but it seems that it will take some time because I'm not familiar with HDF5.

I wonder what the questions about gradient descent and gradient boost are asking. Could you give me a little more detail?

[Mec-iS]

Thanks for the great presentation of peroxide, there are so many good points, in particular the Dataframe and Series implementation that we don't have yet in smartcore

I wonder what the questions about gradient descent and gradient boost are asking. Could you give me a little more detail?

Sure. as XGBoost is the most popular algorithm for gradient descent, we are considering having it implemented in smartcore, see this for a reference.
What are in your opinion the most relevant use cases for a linear algebra user to use XGBoost? How would you envision a good interface to allow usage of gradient descent in a linear algebra library (not involving Neural Nets)?

[Axect]

Well... first of all, I don't have a special idea about XGBoost because I don't use Boosted Decision Tree methods well.
However, in the case of gradient descent, there are a few things to say about the necessity and implementation.

In my case, there were quite a few cases where gradient calculation was needed for arbitrary functions when performing optimization (e.g. Newton's method, etc.). Therefore, in peroxide, there were some automatic differentiation tools - peroxide-ad.
I use this to compute exact jacobian or hessian and use these in newton's method. Here is the example - Root finding via AD. Or even I wrote a skeleton multi-layer perceptron with it - MLP (from scratch), but that's probably not what you're interested in.

Since I'm not familiar with gradient boosting, this is all my opinion. I apologize if the focus is off 😥

[Mec-iS]

Thanks, this is a good answer that clears some of my doubts.

[Mec-iS]

Is there something you would like to be helped with to improve peroxide?

EDIT: it would be possible to use peroxide to implement RL algorithms (see #244 )? How would you proceed?