Describe the bug

In grid_to_graph, you expect the vertices to correspond to the implicit order defined by the mask. This is not always the case, due to the occurrence of isolated vertices that are dismissed in the reindexing of the vertices.

Steps/Code to Reproduce

import numpy as np
from sklearn.feature_extraction import grid_to_graph

mask = np.zeros((2, 3)).astype('bool')
mask[0, 0] = 1
mask[0, 2] = 1
mask[1, 2] = 1
print(grid_to_graph(2, 3, 1, mask.ravel()).todense())

Expected Results

[[1 0 0]
[0 1 1]
[0 1 1]]

Actual Results

[[1 1 0]
[1 1 0]
[0 0 1]]

Versions

System:
machine: Linux-4.4.0-187-generic-x86_64-with-Ubuntu-16.04-xenial
python: 3.5.2 (default, Oct 7 2020, 17:19:02) [GCC 5.4.0 20160609]
executable: /usr/bin/python3

BLAS:
lib_dirs: /usr/lib
macros: HAVE_CBLAS=None
cblas_libs: openblas, openblas

Python deps:
scipy: 1.1.0
Cython: 0.29
pip: 18.1
pandas: 0.23.4
sklearn: 0.22.dev0
setuptools: 40.4.3
numpy: 1.15.4