机器学习入门之(原创)（四）机器学习笔记之Scikit Learn的Logistic回归初探-职坐标

机器学习入门之(原创)（四）机器学习笔记之Scikit Learn的Logistic回归初探

小标 2018-10-15 来源：阅读 1252 评论 0

摘要：本文主要向大家介绍了机器学习入门之(原创)（四）机器学习笔记之Scikit Learn的Logistic回归初探，通过具体的内容向大家展现，希望对大家学习机器学习入门有所帮助。

本文主要向大家介绍了机器学习入门之(原创)（四）机器学习笔记之Scikit Learn的Logistic回归初探，通过具体的内容向大家展现，希望对大家学习机器学习入门有所帮助。

目录
一、Scikit Learn中有关logistics回归函数的介绍... 2
1. 交叉验证... 2
2. 使用搜索进行正则化的 Logistic Regression参数调优... 3
3. 用LogisticRegressionCV实现正则化的 Logistic Regression 参数调优... 6
二、应用举例... 10
1.    读取数据... 10
2.    看各类样本分布是否均衡... 10
3.    特征编码... 10
4.    数据预处理... 10
5.    模型训练... 10
5.1 交叉验证进行 Logistic Regression 参数调优... 10
5.2 使用搜索进行正则化的 Logistic Regression参数调优... 10
5.3 使用LogisticRegressionCV进行正则化的 Logistic Regression 参数调优
... 10

一、Scikit Learn中有关logistics回归函数的介绍
1.
交叉验证
交叉验证用于评估模型性能和进行参数调优（模型选择）。分类任务中交叉验证缺省是采用StratifiedKFold。

sklearn.cross_validation.cross_val_score(estimator,
X, y=None, scoring=None, cv=None, n_jobs=1, verbose=0, fit_params=None,
pre_dispatch=‘2*n_jobs‘)

Parameters:

estimator : estimator object implementing
‘fit’
The object to use to fit the data.
X : array-like
The data to fit. Can be, for example a  list, or an
array at least 2d.
y : array-like, optional, default: None
The target variable to try to predict  in the case of
supervised learning.
scoring : string, callable or None,  optional, default:
None
A string (see model evaluation  documentation) or a
scorer callable object / function with signature scorer(estimator, X, y).
cv : int, cross-validation generator or an iterable,
optional
Determines the cross-validation splitting strategy.
Possible inputs for cv are:

None, to use the default 3-fold
cross-validation,
integer, to specify the number of folds.
An object to be used as a cross-validation
generator.
An iterable yielding train/test splits.

For integer/None inputs, if the estimator is a
classifier and y is either binary or  multiclass, StratifiedKFold is
used. In all  other cases, KFold is
used.
Refer User Guide for  the various
cross-validation strategies that can be used here.
n_jobs : integer, optional
The number of CPUs to use to do the  computation. -1
means ‘all CPUs’.
verbose : integer, optional
The verbosity level.
fit_params : dict, optional
Parameters to pass to the fit method  of the
estimator.
pre_dispatch : int, or string, optional
Controls the number of jobs that get dispatched during
parallel execution. Reducing this number can be useful to avoid an explosion  of
memory consumption when more jobs get dispatched than CPUs can process.  This
parameter can be:
·         None, in which case all the jobs are immediately
created and spawned. Use this for lightweight and fast-running jobs, to avoid
delays due to on-demand spawning of the jobs
·         An int, giving the exact number of total jobs that are
spawned
·         A string, giving an expression as a function of n_jobs,
as in ‘2*n_jobs’

Returns:

scores : array of float,
shape=(len(list(cv)),)
Array of scores of the estimator for  each run of the
cross validation.

2.
使用搜索进行正则化的 Logistic Regression参数调优

sklearn.grid_search.GridSearchCV(estimator,
param_grid, scoring=None, fit_params=None, n_jobs=1, iid=True, refit=True,
cv=None, verbose=0, pre_dispatch=‘2*n_jobs‘, error_score=‘raise‘)

Parameters:

estimator : estimator object.
A object of that type is instantiated  for each grid
point. This is assumed to implement the scikit-learn estimator  interface.
Either estimator needs to provide a score function,  or scoring must be  passed.
param_grid : dict or list of dictionaries
Dictionary with parameters names  (string) as keys and
lists of parameter settings to try as values, or a list  of such dictionaries,
in which case the grids spanned by each dictionary in  the list are explored.
This enables searching over any sequence of parameter  settings.
scoring : string, callable or None,
default=None
A string (see model evaluation  documentation) or a
scorer callable object / function with signature scorer(estimator, X, y). If None, the score method  of the estimator is used.
fit_params : dict, optional
Parameters to pass to the fit method.
n_jobs : int, default=1
Number of jobs to run in parallel.
Changed in version 0.17: Upgraded to  joblib 0.9.3.
pre_dispatch : int, or string, optional
Controls the number of jobs that get dispatched during
parallel execution. Reducing this number can be useful to avoid an explosion  of
memory consumption when more jobs get dispatched than CPUs can process.  This
parameter can be:
·         None, in which case all the jobs are immediately
created and spawned. Use this for lightweight and fast-running jobs, to avoid
delays due to on-demand spawning of the jobs
·         An int, giving the exact number of total jobs that are
spawned
·         A string, giving an expression as a function of n_jobs,
as in ‘2*n_jobs’
iid : boolean, default=True
If True, the data is assumed to be  identically
distributed across the folds, and the loss minimized is the total  loss per
sample, and not the mean loss across the folds.
cv : int, cross-validation generator or an iterable,
optional
Determines the cross-validation splitting strategy.
Possible inputs for cv are:

None, to use the default 3-fold
cross-validation,
integer, to specify the number of folds.
An object to be used as a cross-validation
generator.
An iterable yielding train/test splits.

For integer/None inputs, if the estimator is a
classifier and y is either binary or
multiclass, sklearn.model_selection.StratifiedKFold is used.  In all other
cases, sklearn.model_selection.KFold is used.
Refer User Guide for  the various
cross-validation strategies that can be used here.
refit : boolean, default=True
Refit the best estimator with the  entire dataset. If
“False”, it is impossible to make predictions using this  GridSearchCV instance
after fitting.
verbose : integer
Controls the verbosity: the higher,  the more
messages.
error_score : ‘raise’ (default) or numeric
Value to assign to the score if an  error occurs in
estimator fitting. If set to ‘raise’, the error is raised. If  a numeric value
is given, FitFailedWarning is raised. This parameter does not  affect the refit
step, which will always raise the error.

Attributes:

grid_scores_ : list of named tuples
Contains scores for all parameter combinations in
param_grid. Each entry corresponds to one parameter setting. Each named tuple
has the attributes:
·         parameters, a dict of parameter settings
·         mean_validation_score,
the mean  score over the cross-validation folds
·         cv_validation_scores,
the list of  scores for each fold
best_estimator_ :
estimator
Estimator that was chosen by the  search, i.e.
estimator which gave highest score (or smallest loss if specified)  on the left
out data. Not available if refit=False.
best_score_ : float
Score of best_estimator on the left  out
data.
best_params_ : dict
Parameter setting that gave the best  results on the
hold out data.
scorer_ : function
Scorer function used on the held out  data to choose
the best parameters for the model.

训练：

fit(X, y=None)

Run fit with all sets of parameters.

Parameters:

X : array-like, shape = [n_samples,
n_features]
Training vector, where n_samples is the number of
samples and n_features is the number of features.
y : array-like, shape = [n_samples] or  [n_samples,
n_output], optional
Target relative to X for classification or regression;
None for unsupervised learning.

3.
用LogisticRegressionCV实现正则化的 Logistic Regression 参数调优

sklearn.linear_model.LogisticRegressionCV(Cs=10,
fit_intercept=True, cv=None, dual=False, penalty=‘l2‘, scoring=None,
solver=‘lbfgs‘, tol=0.0001, max_iter=100, class_weight=None, n_jobs=1,
verbose=0, refit=True, intercept_scaling=1.0, multi_class=‘ovr‘,
random_state=None)

Parameters:

Cs : list of floats | int
Each of the values in Cs describes the  inverse of
regularization strength. If Cs is as an int, then a grid of Cs  values are
chosen in a logarithmic scale between 1e-4 and 1e4. Like in  support vector
machines, smaller values specify stronger regularization.
fit_intercept :
bool, default: True
Specifies if a constant (a.k.a. bias  or intercept)
should be added to the decision function.
class_weight : dict or ‘balanced’, optional
Weights associated with classes in the
form {class_label: weight}.
If not given, all classes are supposed to have weight  one.
The “balanced” mode uses the values of y to
automatically adjust weights inversely proportional to class frequencies in  the
input data as n_samples / (n_classes * np.bincount(y)).
Note that these weights will be multiplied with
sample_weight (passed through the fit method) if sample_weight is
specified.
New in version 0.17: class_weight
== ‘balanced’
cv : integer or cross-validation
generator
The default cross-validation generator  used is
Stratified K-Folds. If an integer is provided, then it is the number of  folds
used. See the module sklearn.model_selection module
for the list of possible  cross-validation objects.
penalty : str, ‘l1’ or ‘l2’
Used to specify the norm used in the  penalization. The
‘newton-cg’, ‘sag’ and ‘lbfgs’ solvers support only l2  penalties.
dual : bool
Dual or primal formulation. Dual  formulation is only
implemented for l2 penalty with liblinear solver. Prefer  dual=False when
n_samples > n_features.
scoring : callabale
Scoring function to use as  cross-validation criteria.
For a list of scoring functions that can be used,  look at sklearn.metrics.
The default scoring option used is accuracy_score.
solver : {‘newton-cg’, ‘lbfgs’,   ‘liblinear’,
‘sag’}
Algorithm to use in the optimization
problem.

For small datasets,        ‘liblinear’ is a good
choice, whereas ‘sag’ is

faster for large ones.

For multiclass       problems, only ‘newton-cg’, ‘sag’
and ‘lbfgs’ handle

multinomial loss; ‘liblinear’ is limited to
one-versus-rest schemes.

‘newton-cg’, ‘lbfgs’ and ‘sag’ only handle L2
penalty.
‘liblinear’ might be       slower in
LogisticRegressionCV because it does

not handle warm-starting.
Note that ‘sag’ fast convergence is only guaranteed on
features with approximately the same scale. You can preprocess the data with  a
scaler from sklearn.preprocessing.
New in version 0.17: Stochastic
Average Gradient descent solver.
tol : float, optional
Tolerance for stopping criteria.
max_iter : int, optional
Maximum number of iterations of the  optimization
algorithm.
n_jobs : int, optional
Number of CPU cores used during the  cross-validation
loop. If given a value of -1, all cores are used.
verbose : int
For the ‘liblinear’, ‘sag’ and ‘lbfgs’   solvers set
verbose to any positive number for verbosity.
refit : bool
If set to True, the scores are  averaged across all
folds, and the coefs and the C that corresponds to the  best score is taken, and
a final refit is done using these parameters.  Otherwise the coefs, intercepts
and C that correspond to the best scores  across folds are averaged.
multi_class : str, {‘ovr’, ‘multinomial’}
Multiclass option can be either ‘ovr’ or ‘multinomial’.
  If the option chosen is ‘ovr’, then a binary problem is fit for each label.
Else the loss minimised is the multinomial loss fit across the entire
probability distribution. Works only for the ‘newton-cg’, ‘sag’ and ‘lbfgs’
solver.
New in version 0.18: Stochastic
Average Gradient descent solver for ‘multinomial’ case.
intercept_scaling :
float, default 1.
Useful only when the solver ‘liblinear’ is used and
self.fit_intercept is set to True. In this case, x becomes [x,
self.intercept_scaling], i.e. a “synthetic” feature with constant value equal
to intercept_scaling is appended to the instance vector. The intercept
becomes intercept_scaling * synthetic_feature_weight.
Note! the synthetic feature weight is  subject to l1/l2
regularization as all other features. To lessen the effect  of regularization on
synthetic feature weight (and therefore on the  intercept) intercept_scaling has
to be increased.
random_state : int seed, RandomState  instance, or None
(default)
The seed of the pseudo random number  generator to use
when shuffling the data.

Attributes:

coef_ : array, shape (1, n_features) or (n_classes,
n_features)
Coefficient of the features in the decision
function.
coef_ is of shape (1, n_features)  when the given
problem is binary. coef_ is readonly property  derived
from raw_coef_ that follows the internal memory layout of
liblinear.
intercept_ : array, shape (1,) or (n_classes,)
Intercept (a.k.a. bias) added to the  decision
function. It is available only when parameter intercept is set to  True and is
of shape(1,) when the problem is binary.
Cs_ : array
Array of C i.e. inverse of  regularization parameter
values used for cross-validation.
coefs_paths_ : array, shape (n_folds, len(Cs_), n_features) or (n_folds, len(Cs_), n_features + 1)
dict with classes as the keys, and the  path of
coefficients obtained during cross-validating across each fold and  then across
each Cs after doing an OvR for the corresponding class as values.  If the
‘multi_class’ option is set to ‘multinomial’, then the coefs_paths are  the
coefficients corresponding to each class. Each dict value has
shape (n_folds, len(Cs_), n_features) or (n_folds, len(Cs_), n_features + 1) depending  on whether the intercept is fit or
not.
scores_ : dict
dict with classes as the keys, and the  values as the
grid of scores obtained during cross-validating each fold,  after doing an OvR
for the corresponding class. If the ‘multi_class’ option  given is ‘multinomial’
then the same scores are repeated across all classes,  since this is the
multinomial class. Each dict value has shape (n_folds,  len(Cs))
C_ : array, shape (n_classes,) or (n_classes -
1,)
Array of C that maps to the best  scores across every
class. If refit is set to False, then for each class, the  best C is the average
of the C’s that correspond to the best scores for each  fold.
n_iter_ : array, shape (n_classes, n_folds,  n_cs) or (1,
n_folds, n_cs)
Actual number of iterations for all  classes, folds and
Cs. In the binary or multinomial cases, the first  dimension is equal to
1.

二、应用举例
Kaggle 2015年举办的Otto Group Product Classification Challenge竞赛数据为例。
1.   读取数据
# 首先 import 必要的模块
import pandas as pd
import numpy as np

from sklearn.model_selection import GridSearchCV

#评价指标为logloss
from sklearn.metrics import log_loss

from matplotlib import pyplot
import seaborn as sns
%matplotlib inline

# 读取数据
dpath = ‘./data/‘
train = pd.read_csv(dpath +"Otto_train.csv")
train.head()

2.   看各类样本分布是否均衡

# Target 分布，看看各类样本分布是否均衡
sns.countplot(train.target);
pyplot.xlabel(‘target‘);
pyplot.ylabel(‘Number of occurrences‘);

各类样本不均衡。交叉验证对分类任务缺省的是采用StratifiedKFold，在每折采样时根据各类样本按比例采样

3.   特征编码
# 将类别字符串变成数字
y_train = train[‘target‘]
y_train = y_train.map(lambda s: s[6:])      # 对于s使用s[6:]来代替
y_train = y_train.map(lambda s: int(s)-1)   # 对于s使用int(s)-1来代替

train = train.drop(["id", "target"], axis=1) # 去掉 "id", "target"这2列
X_train = np.array(train)[0:2000,:] # 转为数组

4.   数据预处理
# 数据标准化
from sklearn.preprocessing import StandardScaler

# 初始化特征的标准化器
ss_X = StandardScaler()

# 分别对训练数据的特征进行标准化处理
X_train = ss_X.fit_transform(X_train)

5.   模型训练
5.1 交叉验证进行 Logistic Regression 参数调优

from sklearn.linear_model import LogisticRegression
lr= LogisticRegression()

# 交叉验证用于评估模型性能和进行参数调优（模型选择）
#分类任务中交叉验证缺省是采用StratifiedKFold
from sklearn.cross_validation import cross_val_score
# cross_val_score(estimator, X, y=None, scoring=None, cv=None, ...)
# estimator: 模型，X：特征，y:标签，scoring：分数规则，cv：k折交叉验证
scores = cross_val_score(lr, X_train, y_train, cv=5, scoring=‘accuracy‘)
print ‘accuracy of each fold is: ‘
print(scores)
print‘cv accuracy is:‘, scores.mean()

accuracy of each fold is:
[ 0.97755611  0.9925      0.9775      0.9875      0.98746867]
cv accuracy is: 0.984504956281

5.2 使用搜索进行正则化的 Logistic Regression参数调优
from sklearn.model_selection import GridSearchCV
from sklearn.linear_model import LogisticRegression

#需要调优的参数
# 请尝试将L1正则和L2正则分开，并配合合适的优化求解算法（slover）
#tuned_parameters = {‘penalty‘:[‘l1‘,‘l2‘],
#                   ‘C‘: [0.001, 0.01, 0.1, 1, 10, 100, 1000]
#                   }
penaltys = [‘l1‘,‘l2‘]
Cs = [0.001, 0.01, 0.1, 1, 10, 100, 1000]
tuned_parameters = dict(penalty = penaltys, C = Cs)

lr_penalty= LogisticRegression()
# GridSearchCV(estimator, param_grid, ... cv=None, ...)
# estimator: 模型， param_grid：字典类型的参数， cv：k折交叉验证
grid= GridSearchCV(lr_penalty, tuned_parameters, cv=5)
grid.fit(X_train,y_train) # 网格搜索训练

grid.cv_results_ #训练的结果

{‘mean_fit_time‘: array([ 0.00779996,  0.01719995,  0.01200004,  0.02780004,  0.01939998,
         0.03739996,  0.048     ,  0.05899997,  0.21480007,  0.12020001,
         0.4348001 ,  0.13859997,  0.39040003,  0.15320001]),
‘mean_score_time‘: array([ 0.00039997,  0.00040002,  0.00059996,  0.00059996,  0.00059996,
         0.0006    ,  0.00039997,  0.00019999,  0.00040002,  0.00040002,
         0.0006    ,  0.0006    ,  0.00079994,  0.00099993]),
‘mean_test_score‘: array([ 0.9645,  0.976 ,  0.9645,  0.9805,  0.9785,  0.9805,  0.985 ,
         0.9845,  0.983 ,  0.9805,  0.98  ,  0.977 ,  0.9775,  0.974 ]),
‘mean_train_score‘: array([ 0.96450007,  0.98012508,  0.96512492,  0.98399976,  0.98137492,
         0.987875  ,  0.99462492,  0.9945    ,  0.999625  ,  0.99824992,
         1.        ,  1.        ,  1.        ,  1.        ]),
‘param_C‘: masked_array(data = [0.001 0.001 0.01 0.01 0.1 0.1 1 1 10 10 100 100 1000 1000],
              mask = [False False False False False False False False False False False False
  False False],
        fill_value = ?),
‘param_penalty‘: masked_array(data = [‘l1‘ ‘l2‘ ‘l1‘ ‘l2‘ ‘l1‘ ‘l2‘ ‘l1‘ ‘l2‘ ‘l1‘ ‘l2‘ ‘l1‘ ‘l2‘ ‘l1‘ ‘l2‘],
              mask = [False False False False False False False False False False False False
  False False],
        fill_value = ?),
‘params‘: ({‘C‘: 0.001, ‘penalty‘: ‘l1‘},
  {‘C‘: 0.001, ‘penalty‘: ‘l2‘},
  {‘C‘: 0.01, ‘penalty‘: ‘l1‘},
  {‘C‘: 0.01, ‘penalty‘: ‘l2‘},
  {‘C‘: 0.1, ‘penalty‘: ‘l1‘},
  {‘C‘: 0.1, ‘penalty‘: ‘l2‘},
  {‘C‘: 1, ‘penalty‘: ‘l1‘},
  {‘C‘: 1, ‘penalty‘: ‘l2‘},
  {‘C‘: 10, ‘penalty‘: ‘l1‘},
  {‘C‘: 10, ‘penalty‘: ‘l2‘},
  {‘C‘: 100, ‘penalty‘: ‘l1‘},
  {‘C‘: 100, ‘penalty‘: ‘l2‘},
  {‘C‘: 1000, ‘penalty‘: ‘l1‘},
  {‘C‘: 1000, ‘penalty‘: ‘l2‘}),
‘rank_test_score‘: array([13, 11, 13,  4,  8,  4,  1,  2,  3,  4,  7, 10,  9, 12]),
‘split0_test_score‘: array([ 0.96259352,  0.96758105,  0.96259352,  0.97506234,  0.97256858,
         0.97506234,  0.97755611,  0.97755611,  0.97755611,  0.98004988,
         0.97007481,  0.97256858,  0.97506234,  0.97506234]),
‘split0_train_score‘: array([ 0.96497811,  0.98186366,  0.9656035 ,  0.98373984,  0.98186366,
         0.98811757,  0.99437148,  0.99437148,  0.99937461,  0.99749844,
         1.        ,  1.        ,  1.        ,  1.        ]),
‘split1_test_score‘: array([ 0.965 ,  0.9825,  0.965 ,  0.9875,  0.9825,  0.9875,  0.99  ,
         0.9925,  0.9825,  0.9825,  0.9775,  0.9825,  0.9725,  0.9775]),
‘split1_train_score‘: array([ 0.964375,  0.97625 ,  0.965   ,  0.983125,  0.979375,  0.986875,
         0.994375,  0.994375,  0.999375,  0.996875,  1.      ,  1.      ,
         1.      ,  1.      ]),
‘split2_test_score‘: array([ 0.965 ,  0.9825,  0.965 ,  0.9825,  0.9825,  0.9825,  0.9825,
         0.9775,  0.9775,  0.97  ,  0.9775,  0.9675,  0.975 ,  0.965 ]),
‘split2_train_score‘: array([ 0.964375,  0.980625,  0.964375,  0.98375 ,  0.981875,  0.98875 ,
         0.99625 ,  0.995625,  1.      ,  0.999375,  1.      ,  1.      ,
         1.      ,  1.      ]),
‘split3_test_score‘: array([ 0.965 ,  0.9775,  0.965 ,  0.9825,  0.985 ,  0.9825,  0.99  ,
         0.9875,  0.9875,  0.985 ,  0.985 ,  0.98  ,  0.985 ,  0.975 ]),
‘split3_train_score‘: array([ 0.964375,  0.980625,  0.964375,  0.98375 ,  0.98125 ,  0.9875  ,
         0.993125,  0.99375 ,  1.      ,  0.999375,  1.      ,  1.      ,
         1.      ,  1.      ]),
‘split4_test_score‘: array([ 0.96491228,  0.96992481,  0.96491228,  0.97493734,  0.96992481,
         0.97493734,  0.98496241,  0.98746867,  0.98997494,  0.98496241,
         0.98997494,  0.98245614,  0.97994987,  0.97744361]),
‘split4_train_score‘: array([ 0.96439725,  0.98126171,  0.96627108,  0.98563398,  0.98251093,
         0.98813242,  0.99500312,  0.99437851,  0.99937539,  0.99812617,
         1.        ,  1.        ,  1.        ,  1.        ]),
‘std_fit_time‘: array([ 0.0011662 ,  0.00116623,  0.00063249,  0.00305936,  0.00185475,
         0.00205906,  0.00460443,  0.0018974 ,  0.04810566,  0.01215555,
         0.17968574,  0.01993598,  0.10196788,  0.02121689]),
‘std_score_time‘: array([ 0.00048986,  0.00048992,  0.00048986,  0.00048986,  0.00048986,
         0.0004899 ,  0.00048986,  0.00039997,  0.00048992,  0.00048992,
         0.0004899 ,  0.0004899 ,  0.00039997,  0.        ]),
‘std_test_score‘: array([ 0.00095533,  0.00623894,  0.00095533,  0.00484784,  0.00604764,
         0.00484784,  0.00472867,  0.00598547,  0.00507914,  0.00555997,
         0.00686303,  0.00598133,  0.00445968,  0.00463055]),
‘std_train_score‘: array([ 0.00023917,  0.00199152,  0.00073254,  0.00085184,  0.00107655,
         0.00063739,  0.00101591,  0.00061238,  0.00030619,  0.00100021,
         0.        ,  0.        ,  0.        ,  0.        ])}

# examine the best model
print(grid.best_score_)  # 最好的分数
print(grid.best_params_) # 最好的参数

0.754775526035
{‘penalty‘: ‘l1‘, ‘C‘: 100}

如果最佳值在候选参数的边缘，最好再尝试更大的候选参数或更小的候选参数，直到找到拐点。

5.3 使用LogisticRegressionCV进行正则化的 Logistic Regression 参数调优

from sklearn.linear_model import LogisticRegressionCV

Cs = [1, 10,100,1000]

# 大量样本（7W）、高维度（93），L1正则 --> 可选用saga优化求解器(0.19版本新功能)
lr_cv = LogisticRegressionCV(Cs=Cs, cv = 5, penalty=‘l1‘, solver=‘liblinear‘, multi_class=‘ovr‘)
lr_cv.fit(X_train, y_train)

LogisticRegressionCV(Cs=[1, 10, 100, 1000], class_weight=None, cv=5,
           dual=False, fit_intercept=True, intercept_scaling=1.0,
           max_iter=100, multi_class=‘ovr‘, n_jobs=1, penalty=‘l1‘,
           random_state=None, refit=True, scoring=None, solver=‘liblinear‘,
           tol=0.0001, verbose=0)

lr_cv.scores_ # 网格中每次迭代的分数

{1: array([[ 0.97755611,  0.97755611,  0.97007481,  0.97506234],
        [ 0.99      ,  0.9825    ,  0.9775    ,  0.975     ],
        [ 0.9825    ,  0.9775    ,  0.9775    ,  0.975     ],
        [ 0.99      ,  0.9875    ,  0.985     ,  0.985     ],
        [ 0.98496241,  0.98997494,  0.98997494,  0.97994987]])}