pCMF example gives segmentation fault

[pcarbo]

Hi @gdurif, I tried to use your R package. I was able to successfully install the package, but I ran into trouble when trying to run the cNMF example provided. This is the error I get:

Declaration
Initialization
Optimization
iter 0
 *** caught segfault ***
address 0x7ffeecfec000, cause 'memory not mapped'

This is the exact code I ran (note that I added a set.seed; without it, your example is not reproducible):

library(pCMF)
set.seed(1)
n <- 100
p <- 500
K <- 20

# SIMULATE DATA
# -------------
factorU <- generate_factor_matrix(n,K,ngroup = 3,
                                  average_signal = c(250,100,250),
                                  group_separation = 0.8,
                                  distribution = "exponential",
                                  shuffle_feature = TRUE)

factorV <- generate_factor_matrix(p,K,ngroup = 2,average_signal = 80,
                                  group_separation = 0.8,
                                  distribution = "exponential",
                                  shuffle_feature = TRUE,
                                  prop_noise_feature = 0.6)

U <- factorU$factor_matrix
V <- factorV$factor_matrix

count_data <- generate_count_matrix(n,p,K,U,V,ZI = TRUE,prob1 = rep(0.3,p))
X          <- count_data$X
kept_cols  <- prefilter(X,prop = 0.05,quant_max = 0.95,
            	        presel = TRUE,threshold = 0.2)
X          <- X[,kept_cols]

# FIT MATRIX FACTORIZATION
# ------------------------
out <- pCMF(X,K,verbose = TRUE,zero_inflation = TRUE,sparsity = TRUE)

Please advise. Note that I am using R 3.4.3 on a MacBook Pro:

> sessionInfo()
R version 3.4.3 (2017-11-30)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS High Sierra 10.13.6

Matrix products: default
BLAS: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] pCMF_1.2.1

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.1          magrittr_1.5        maps_3.3.0
 [4] tidyselect_0.2.5    munsell_0.4.3       colorspace_1.4-0
 [7] lattice_0.20-35     R6_2.4.0            rlang_0.3.1
[10] plyr_1.8.4          dplyr_0.8.0.1       tools_3.4.3
[13] fields_9.6          dotCall64_1.0-0     grid_3.4.3
[16] spam_2.2-2          gtable_0.2.0        lazyeval_0.2.1
[19] assertthat_0.2.1    tibble_2.1.1        RcppEigen_0.3.3.5.0
[22] crayon_1.3.4        Matrix_1.2-12       purrr_0.2.5
[25] ggplot2_3.2.0       glue_1.3.1          compiler_3.4.3
[28] pillar_1.3.1        scales_0.5.0        pkgconfig_2.0.2

[gdurif]

Thank you very much for this report, I am quite busy at the moment, I will try to find the problem later in the day. Thanks again

[pcarbo]

@gdurif Thank you. Please let me know if you cannot reproduce the error.

[gdurif]

I am not able to reproduce the bug (I don't have MacOS to run some tests). Do you know which compiler is used when R install the package and which version of pCMF did you install? (prod or prod_no_omp?)
Thanks

[pcarbo]

@gdurif I re-installed pCMF from the prod_no_omp branch, and that worked. So the bug is specific to your OpenMP implementation. In any case, I am fine to continue to use the non-OpenMP version.

[gdurif]

Ok, I guess the problem is with the compiler. On MacOS, OpenMP support in the compiler can be an issue. I will try to find the problem anyway.

[pcarbo]

@gdurif I'm not using the compiler provided by Apple. I'm using a version of clang installed via MacPorts:

CC       = /opt/local/bin/clang-mp-4.0
CXX      = /opt/local/bin/clang++-mp-4.0
CXX11    = /opt/local/bin/clang++-mp-4.0
CXX17    = /opt/local/bin/clang++-mp-4.0
CXX98    = /opt/local/bin/clang++-mp-4.0
CXX1X    = /opt/local/bin/clang++-mp-4.0
F77      = /opt/local/bin/gfortran-mp-6
FC       = /opt/local/bin/gfortran-mp-6
LDFLAGS  = -L/opt/local/lib # -L/usr/local/Cellar/hdf5/1.10.1_2/lib
FLIBS    = -L/opt/local/lib/gcc6 -lgfortran -lquadmath -lm

See here for details.

[gdurif]

Thanks for this information.

[l-modolo]

Hi, I don't think that a MacOS problem. Your example run smoothly on :

> sessionInfo()
R version 3.6.1 (2019-07-05)
Platform: x86_64-apple-darwin18.6.0 (64-bit)
Running under: macOS Catalina 10.15.1

Matrix products: default
BLAS/LAPACK: /usr/local/Cellar/openblas/0.3.7/lib/libopenblasp-r0.3.7.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] pCMF_1.2.1

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.3          pillar_1.4.2        compiler_3.6.1
 [4] prettyunits_1.0.2   remotes_2.1.0       tools_3.6.1
 [7] testthat_2.3.0      digest_0.6.22       pkgbuild_1.0.6
[10] dotCall64_1.0-0     pkgload_1.0.2       tibble_2.1.3
[13] memoise_1.1.0       lifecycle_0.1.0     gtable_0.3.0
[16] lattice_0.20-38     pkgconfig_2.0.3     rlang_0.4.1
[19] Matrix_1.2-17       cli_1.1.0           curl_3.3
[22] spam_2.4-0          dplyr_0.8.3         withr_2.1.2
[25] desc_1.2.0          fs_1.3.1            fields_10.0
[28] devtools_2.1.0      maps_3.3.0          tidyselect_0.2.5
[31] rprojroot_1.3-2     grid_3.6.1          glue_1.3.1
[34] R6_2.4.1            processx_3.4.1      sessioninfo_1.1.1
[37] purrr_0.3.2         callr_3.3.2         ggplot2_3.2.1
[40] magrittr_1.5        backports_1.1.5     scales_1.1.0
[43] ps_1.3.0            usethis_1.5.1       assertthat_0.2.1
[46] colorspace_1.4-1    lazyeval_0.2.2      munsell_0.5.0
[49] crayon_1.3.4        RcppEigen_0.3.3.7.0

my Makeconf:

AR = ar
BLAS_LIBS = -L$(R_HOME)/lib$(R_ARCH) -lRblas
C_VISIBILITY =
CC = gcc-8
CFLAGS = -Wall -g $(LTO)
CPICFLAGS = -fPIC
CPPFLAGS = -I/usr/local/include
CXX = gcc-8
CXXCPP = $(CXX) -E
CXXFLAGS = -Wall -g $(LTO)
CXXPICFLAGS = -fPIC
CXX98 = gcc-8
CXX98FLAGS = -Wall -g -O2
CXX98PICFLAGS = -fPIC
CXX98STD =
CXX11 = gcc-8
CXX11FLAGS = -Wall -g -O2
CXX11PICFLAGS = -fPIC
CXX11STD = -std=gnu++11
CXX14 = gcc-8
CXX14FLAGS = -Wall -g -O2
CXX14PICFLAGS = -fPIC
CXX14STD = -std=gnu++14
CXX17 = gcc-8
CXX17FLAGS = -Wall -g -O2
CXX17PICFLAGS = -fPIC
CXX17STD = -std=gnu++1z

[pcarbo]

@l-modolo Thanks for that info. Your compiler setup isn't the one recommended by CRAN (you should be using clang), but glad to know that it can work on macOS.