Error in La.svd(x, nu, nv) : error code 1 from Lapack routine 'dgesdd'

[cizydorczyk]

Hello,

I am trying to run DAPC cross-validation using the xvalDapc() function. The data set I am working with is a matrix of 155 rows and ~98K columns representing SNP presence/absence data. Unfortunately, when I try to run the cross-validation, I get the following error:

> set.seed(42)
> xval.pop <- xvalDapc(snps, pca.clusters.factor)
Error in La.svd(x, nu, nv) : error code 1 from Lapack routine 'dgesdd'
In addition: Warning message:
In xvalDapc.data.frame(snps, pca.clusters.factor) :
  8 groups have only 1 member: these groups cannot be represented in both training and validation sets.

The main part I am having issues here is the first error "Error in la.svd(x, nu, nv)..." The second warning message I understand and am aware that some clusters contain only one isolate.

From my understanding, this error is saying something about single value decomposition failing/not converging? The pca.clusters.factor is a factor containing the prior groupings I am using. It contains 31 clusters, with varying numbers of isolates per group. These clusters were created using hierarchical clustering of principal components, not the find.clusters() function. I thought that might have been the cause for the error, but I have tried the exact same procedure using a different SNP dataset of similar size and a separate clusters factor (also generated using hierarchical clustering of principal components) with no such errors.

Unfortunately, I do not have much background in dealing with matrix operations, so I am really unsure why svd might be failing.

Any help in resolving this issue would be much appreciated.

Thank you,
Conrad Izydorczyk

[caitiecollins]

Hey Conrad,

I'm afraid I'm not entirely sure what's causing this problem for you.

You're correct that the warning message should be there, as you have single-individual clusters and xval won't be able to do anything with these (nor, for that matter, will it be especially able to do very useful things with small clusters like n=2 -- just something to consider.)

Regarding your svd error. I believe svd is being called within the lda function (in the dapc function, in the xval function). The type of clustering approach you took should not be the problem, as long as the pop factor is in the right format. Thibaut or other adegenet contributors will likely know more about what causes this svd error.

In the meantime, have you tried this potential fix? If not, it seems to have worked for others getting this error, and all it takes is to run these lines of code:

## Store the svd function, but with LINPACK = T as default:
svd <- function (x, nu = min(n, p), nv = min(n, p), LINPACK = TRUE)
{
  print("LINPACK:"); print(LINPACK)  ## added so you can see it's changed
  x <- as.matrix(x)
  if (any(!is.finite(x)))
    stop("infinite or missing values in 'x'")
  dx <- dim(x)
  n <- dx[1L]
  p <- dx[2L]
  if (!n || !p)
    stop("a dimension is zero")
  La.res <- La.svd(x, nu, nv)   ## your problem line
  res <- list(d = La.res$d)
  if (nu)
    res$u <- La.res$u
  if (nv) {
    if (is.complex(x))
      res$v <- Conj(t(La.res$vt))
    else res$v <- t(La.res$vt)
  }
  res
}

## Over-write current svd fn with new version:
assignInNamespace("svd", svd, "base")

## Run your xval line to see if it worked:
set.seed(42)
xval.pop <- xvalDapc(snps, pca.clusters.factor)

If that doesn't resolve the issue, hopefully someone else will be able to chime in with a more specific solution shortly.

Cheers,
Caitlin.

[ghanesh]

Hi there,

I have the same issue when running the ordiR2step function from vegan.
This is a traceback
13. La.svd(x, nu, nv)
12. svd(Y)
11. ordResid(Y)
10. ordConstrained(X, Y, Z, arg = scale, method = "rda")
9. rda.default(d$X, d$Y, d$Z, scale)
8. rda.formula(formula = data ~ 'var1' + 'var2' + 'var3' + 'var4' +'var5' +'var6' + 'var7', data = varlist1)
7. rda.formula(formula = data ~ 'var1' + 'var2' + 'var3' + 'var4' +'var5' +'var6' + 'var7', data = varlist1)
6. eval(call, parent.frame())
5. eval(call, parent.frame())
4. update.default(object, fla)
3. update(object, fla)
2. RsquareAdj(update(object, fla), permutations = R2permutations, ...)

1. ordiR2step(rda(data ~ 1, data = varlist1), scope = formula(rda_varlist1), direction = "forward", steps = 1000, trace = 1)

It happens for some variables when they are added to the model. This seems to be dependent on the order in which they are computed also. Why do I think so? The issue only occurs for some runs but not for others for the same variable if I remove a different variable from the varlist1.
Unfortunately, the hack with setting LINPACK = T does not work for me. I also followed Jari Oksanen's suggestion to transform the data, but this did not help either.
Well, at least I don't seem to be the only one having this issue.

Thanks for your help!

[boutrys]

Exact same error for me and unfortunately @caitiecollins suggestions do not work for me neither

Thanks in advance for new solutions

[nerdcommander]

I'm late to this thread, and my comment isn't strictly related to this package, but when I googled this error for other reasons, this was one of the only useful pieces of info.

I was bootstrapping then doing PCA which causes duplicate samples and, I think, caused problems with the SVD.
The real issue for me (and I think in general) was the SVD used by prcomp() and svd() in R would fail, throw this error, and not produce any results.

LINPACK = T is deprecated in svd() but it was a hint to me that it was something about the linear algebra solver combined with my data that was the issue.

I solved this using the svd R package (https://cran.r-project.org/package=svd) and the propack.svd() function from that package which gives a warning (Invariant subspace) but still produces results up to the point where there is no more variance left. This was good enough for me since I was interested in a low-rank approximation, only needing 10-50 PCs out of several hundred.

TL;DR version, use propack.svd() from the svd() package in R and maybe it will work for what you're doing? @caitiecollins solution of writing over the base svd() function might be necessary if you don't have the option to swap svd functions as I did.

hope this helps...

[Zepeng-Mu]

My problem was unrelated with this either, but is due to wrong openblas/lapack library that R is using after I made some changes to my conda environment.

[laylaeb]

Hi,

I was wondering if someone was able to solve the issue. Got the same error as @ghanesh when running ordistep...
Anyone knows how to proceed? It would be much appreciated.

Thanks!

[DylanDijk]

@Zepeng-Mu Do you remember whether you changed your LAPACK library, and if this fixed the problem

[Zepeng-Mu]

sorry it's been so long I totally cannot recall what I did.