(R) Efficient methods and operators for the sparse matrix classes in 'Matrix' (esp. CSR format or "RsparseMatrix")
GPL-3.0 License
MatrixExtra
is an R package which extends the sparse matrix and sparse vector types in the Matrix package, particularly the CSR or RsparseMatrix
formats (row-major), by providing methods that work natively and efficiently on them without converting them to another format along the way, such as slicing (selecting rows/columns) or concatenating by rows/columns, along with replacing some Matrix
methods with more efficient versions, such as multi-threaded <sparse, dense>
matrix multiplications, much faster slicing for all the sparse types, and faster elementwise addition/subtraction/multiplication, among others.
This package is based on code originally written for the rsparse package by Dmitriy Selivanov, and includes the MIT-licensed robin-map library within its source code.
The Matrix
package provides a rich set of sparse matrix and sparse vector classes with many methods and operators so that they could be used as drop-in replacements of base R's matrices. Unfortunately, the whole package is centered around the CSC format (CsparseMatrix
, column-major), and calling methods and operators which in principle should be efficient in CSR or COO formats will imply first converting the whole matrix to CSC format (a slow and inefficient operation which duplicates the data), on which the operation might be less efficient due to the storage order.
(Longer introduction in the vignette)
Examples:
library(Matrix)
X = matrix(c(0,0,2,1, 0,3,0,0, 0,0,0,0), nrow=3, ncol=4, byrow=TRUE)
X = as(X, "RsparseMatrix")
X
3 x 4 sparse Matrix of class "dgRMatrix"
[1,] . . 2 1
[2,] . 3 . .
[3,] . . . .
### This will forcibly convert the matrix to triplets
X[1:2, ]
2 x 4 sparse Matrix of class "dgTMatrix"
[1,] . . 2 1
[2,] . 3 . .
### This will forcibly convert the matrix to CSC
rbind(X, X)
6 x 4 sparse Matrix of class "dgCMatrix"
[1,] . . 2 1
[2,] . 3 . .
[3,] . . . .
[4,] . . 2 1
[5,] . 3 . .
[6,] . . . .
### This will forcibly convert the matrix to CSC
X * X
3 x 4 sparse Matrix of class "dgCMatrix"
[1,] . . 4 1
[2,] . 9 . .
[3,] . . . .
The CSR sparse format is particularly useful when dealing with machine learning applications - e.g. splitting between a train and test set, tokenizing text features, multiplying a matrix by a vector of coefficients, calculating a gradient observation-by-observation, among others. Many stochastic optimization techniques and libraries (e.g. LibSVM, VowpalWabbit) require the inputs to be in CSR format or alike (see also readsparse), which does not play well with the column-centric methods of Matrix
.
In principle, one could stick with just the CSC format from Matrix
and keep a mental map of the matrix as being transposed. This however gets complicated rather soon and is very prone to errors. Additionally, one might want to pass sparse matrices to another package whose code is outside of one's control, for which the storage format can make a large difference in performance.
Note: This package greatly benefits from extra optimizations that aren't enabled by default for R packages. See this guide for instructions on how to enable them.
install.packages("MatrixExtra")
Documentation is available on CRAN.
A package vignette is available here (and in CRAN).
%*%
), crossprod
and tcrossprod
for many <sparse,dense>
and <dense,sparse>
types, including those from the float package.[
) CSR matrices, along with faster slicing of CSC and COO, and slicing with sparse vectors.rbind(CSR, CSR)
and cbind(CSR, CSR)
).<RsparseMatrix, RsparseMatrix>
and some <RsparseMatrix, TsparseMatrix>
and <sparse, dense>
types, such as +
, -
, *
, &
, |
.sqrt(CSR)
or CSR * scalar
.CSR
/COO
and vectors, such as CSR * vector
, CSR ^ vector
, etc., which can mimick all the quirks of base R if needed.Matrix
(e.g. matrix
-> ngRMatrix
or dgRMatrix
-> lgCMatrix
).mapSparse
, emptySparse
, and a shorter show
method for sparse objects.[<-
).float32
and sparseVector
.Matrix
.Matrix
.library(Matrix)
library(MatrixExtra)
X = matrix(c(0,0,2,1, 0,3,0,0, 0,0,0,0), nrow=3, ncol=4, byrow=TRUE)
X = as(X, "RsparseMatrix")
show(X)
options("MatrixExtra.quick_show" = FALSE)
show(X)
X[1:2, ]
X + X
X * X
X %*% 1:4
X * 1:3
rbind(X, X)
cbind(X, X)
sqrt(X)
diag(X)
as(as.matrix(X), "dgRMatrix")
as.csc.matrix(X)
### New and optional
set_new_matrix_behavior()
inherits(X[1,,drop=TRUE], "sparseVector")
inherits(t(X), "CsparseMatrix")
restore_old_matrix_behavior()