不均衡数据集的分类模型学习方法

简介

不均衡数据集指的是在分类问题中,数据集的两个类别之间的数量差别到达100:1,1000:1甚至10000:1的情况。当然,有些时候在多分类问题(multiclass classification)中也会出现。

如何在不均衡的数据集(imbalanced data)下进行学习,是一个很重要的问题,原因是这样的数据集会很大程度上限制我们的分类模型的准确性,正如文章所说——

“大部分标准的分类算法都期待一个均衡的类别分布,以及相近的误分类损失”。

在不均衡数据集的研究中,主要的领域有医疗分析、欺诈检测、网络入侵识别、溢油检测等。一个非常经典的数据集是乳腺摄影数据集(Mammography Data Set),在这个数据集中,有若干医学摄影图片以及其对应的二分类标签「健康」,「癌性」。

Image result for mammography data set
Mammography Data Set

在这个数据集中,有10923例阴性以及260例阳性。使用传统分类算法,非常容易出现的情况就是,阴性以几乎100%的正确率被成功分类,但是阳性的分类正确性却低至10%以下,在这个情境下,就是多达234例阳性样本被误诊。

在这一类数据集下,我们的目标就变成了在不严重损害大类(major class)的准确性的基础上,尽量让小类(minor class)的分类准确性提高。

不均衡问题的特性

内在不均衡和外在不均衡

数据集不均衡的成因可能是内在的(intrinsic)或者外在的(extrinsic),如之前所述的肿瘤识别,由于样本在真实世界的分布本身就是不均匀的,所以是内在不均衡。

另一方面,还会有一些情况,原本均匀分布的样本数据,由于采样偏差,导致最终进入数据集的时候样本不均匀了,这种情况则被称之为外在不均衡。这个情况最为著名的就是“欺诈检测”数据集。因为真实世界中,那些欺诈者,基本在申请阶段就已经被拒贷了,而我们无法获取到“如果他们借了钱,会不会逾期”,进而无法进入我们的训练数据集。

类间不均衡和类中不均衡

类间不均衡与类中不均衡同样值得注意,正如下图所示,在一些比较复杂的数据上,不仅不同的类别个数相差巨大,同一类样本之间,也会有不同的概念。对于小类别中间的小样本,样本数极少(underrepresented),分类器非常难以学习到完整的分类函数。

图a)只包含了类间不均衡,即五角星表示的“小类”样本个数远小于“圆圈”代表的“大类”,
图b)则同时包含了类间不均衡、类中不均衡、重叠、噪音等等……

小样本问题(small sample size problem)是传统不均衡问题之外的新兴研究方向,讲的是数据的维度和样本的个数显著不均衡。比较有代表性的有人脸识别、基因序列分析等。对于单纯的小样本问题,传统的做法有数据降维等措施。但是倘若小样本问题和类别不均衡同时存在,不仅小类中的样本数量过于稀缺,而且也会导致传统的解决算法无法归纳出有效的分类函数。是以后的研究重点。

不均衡数据集学习的state-of-art算法

当标准的学习算法遇上不均衡数据时,归纳出来的用于描述小类的规则通常比描述大类的规则要少要弱。这既是由于标签数量上的差别,又是由于小类的样本由于数量少本身可信度就较低。

基于采样的算法

Random Sampling

分为随机过采样(random oversampling)和随机欠采样(random undersampling),随机过采样的思路是从小类里面,随机复制若干样本点,加到原来的集合中,使数据集均衡。而随机欠采样则是从大类里面,随机删除若干样本点,使得数据集分布均衡。

随机欠采样可能导致遗失某些重要的信息,而随机过采样则有过拟合风险。

Informed Undersampling

首先讲讲Easy Ensemble算法,这个算法很简单,使用多次随机欠采样,每次都训练一个分类器,然后再将每次欠采样训练出来的分类器组合到一起,形成最终的分类器。

接着是Balance Cascade算法,这个算法是一个迭代算法。

  1. 在大类样本集合S_{maj}中选择一个子集E满足|E| = |S_{min}|
  2. 使用N = {E \cup S_{min}}进行训练,得到分类器
  3. 过滤出N中所有被正确归类的大类中的样本N_{maj}^*
  4. N_{maj}^*中的样本点剔除S_{maj},回到第1步

还有基于KNN的采样算法,该算法使用的四个规则来筛选出比较重要的大类样本点,用这些重要的样本点进行模型的训练。

  • NearMiss-1方法选择与“三个最接近的小类样本”的平均距离最小的大类样本点;
  • NearMiss-2方法选择与“三个最远的少数类样本”的平均距离最小的大类样本点;
  • NearMiss-3方法为每个小类样本选择给定数量的最接近的大类样本点,以确保每个小类样本点都被一些大类样本点包围;
  • “最大距离”方法选择了大多数类的样本点,这些样本点与“平均距离是最近的三个少数类”距离最大。

研究发现,其中NearMiss-2获得的结果较好。

Synthetic Sampling With Data Generation

合成少数子过采样技术(the synthetic minority oversampling technique, SMOTE)对于每个小类样本,寻找同为小类的K近邻,并使用插值法随机选取其连线中间的一个点最为一个新的过采样点。

(1)   \begin{equation*}x_{new} = x_i + (\hat{x_i} - x_i) \times \delta\end{equation*}

具体来说,上式中x_i为小类中的一个样本点,\hat{x_i}x_i的位于小类集合中的K近邻之一,\delta是一个随机[0,1]区间内实数。

Adaptive Synthetic Sampling

先介绍Borderline-SMOTE算法,和普通SMOTE算法不同的是,Borderline-SMOTE算法对于每一个小类样本点,都统计了其最近邻居中大类和小类的样本点个数。

  • 如果小类邻居多于一半,则该点是内部点,记为SAFE
  • 如果小类邻居少于一半,则该点是边界点,记为DANGER
  • 如果小类邻居为0,则该点为噪声,记为NOISE

统计完成后,使用DANGER集合点进行SMOTE的过采样算法。

ADASYN算法在此基础上又做了优化,去掉了“一半”这个人为的常数。使得过采样更加的自然。

(2)   \begin{equation*}G = (|S_{maj}| - |S_{min}|) \times \beta\end{equation*}

这里面\beta \in [0,1]是一个人为设计的常数,G是描述了“究竟小类需要补充多少个样本点?”

接下来,对于每一个小类样本点计算

(3)   \begin{equation*}\Gamma_i = \frac{\Delta_i / K}{Z}\end{equation*}

其中\Delta_i表示样本点的K近邻中有多少属于大类S_{maj}KZ用于归一化,使得\sum_{\Gamma_i} = 1

最终,每一个点按照下面式子,计算需要生成的点的个数,并使用SMOTE算法的方式进行过采样

(4)   \begin{equation*}g_i = \Gamma_i \times G\end{equation*}

Sampling With Data Clean Technique

Tomek link是一个基于数据清洗的技术,适用于一个经过过采样之后的数据集。其目的是为了解决“由于过采样导致的类别存在边界覆盖,进而影响分类器训练”这个问题。

其核心思想是找到两个离的很近的x_i \in  S_{maj}x_j \in S_{min},令d(x_i, x_j)表示x_ix_j的距离,那么(x_i, x_j)被称为Tomek link当且仅当不存在x_k,满足d(x_i, x_k) < d(x_i, x_j)d(x_j, x_k) < d(x_i, x_j)

Tomek link的出现意味着x_ix_j要不然有一个是噪音,要不然就是他们两个位于分类的交界处。将他们同时删除(或删除大类的那一个),可以有效的防止出现类别边界模糊,有利于提高分类器效果。

Cluster-Based Sampling Method

基于聚类的采样算法可以保证类间的不均衡也被关注。

具体来说,该算法首先使用迭代的方式,找到不同的团,这里使用了经典的KNN算法——

  1. 在两个类别中,分别随机选择K个团中心;
  2. 对于每个样本点,都被归类到最近的团中心;
  3. 团中心进行重新计算,用当前所有归类到该团的样本的坐标均值作为新一轮迭代的团中心;
  4. 跳至第2步

接着,将不同的团使用过采样的方式放大到同等规模。如下图所示——

Integration of Sampling and Boosting

SMOTEBoost 算法分为若干轮,每一轮都涉及到重新采样,使用递增的方式建立模型。每一轮,都着重采样哪些不能被上一轮模型所解释的样本。这样,分类器会逐渐习惯去拟合哪些少数的样本,因为他们更大概率会不能被上一轮的模型分类。

对于DataBoost-IM算法来说,则依照一定的比例来采样那些不容易被学习的样本。

基于修改成本函数的算法

一些对于成本敏感的学习算法,使用成本(cost)函数来量化误分类的程度。与其像基于采样的算法构造均衡的数据分布,不如直接设计一个不同的算是度量,使之能够适应不均衡的数据。

Cost-Sensitive Learning Framework

我们定义C(Min, Maj)表示将大类样本误分类为小类的成本,C(Maj, Min)则相反。那么成本敏感(cost-sensitive)学习的目标就是最小化训练集的总体成本,也就是所谓的贝叶斯条件风险(Bayes Conditional Risk)。

这个函数也可以非常容易的扩充到多标签的分类中,具体C矩阵定义如下——

此时,条件风险被定义为

(5)   \begin{equation*}R(i|x) = \sum_i{P(j|x)C(i,j)}\end{equation*}

这里P(j|x)被定义为给定样本x,实际上为类别j的概率。

通常的分类器,都默认分类错误有1的代价,而分类正确没有代价——

(6)   \begin{equation*} C(i,j) = \left\{ \begin{aligned} 1 & (i \neq j) \\ 0 & (i=j) \end{aligned} \right.\end{equation*}

而在非均衡学习中,我们需要保证有C(Maj, Min) > C(Min, Maj),以使得分类器能够更好地分辨大类小类。而接下来的算法,一大特点就是都使用了不同的办法,来讲我们认为定义的成本矩阵放到具体算法实现中——

  • 第一类技术将误分类成本作为数据空间权重应用于数据集;
  • 第二类将成本最小化成本的技巧运用在模型组合上;
  • 最后一类技术将对成本敏感的函数或特性直接纳入分类模型中,以便将对成本敏感的框架拟合到这些分类器的结果中。

Cost-Sensitive Dataspace Weighting with Adaptive Boosting

这里涉及了AdaBoost系列算法,包括AdaBoost.M1,AdaC1,AdaC2,AdaCost,CSB1,CSB2。以AdaBoost.M1为例讲解这类算法的核心思路——

对于数据集迭代,每一轮拥有不同的权值D_{t}(i)

(7)   \begin{equation*}D_{t+1}(i) = D_t(i) exp(-\alpha_t h_t(x_i) y_i) / Z_t\end{equation*}

其中\alpha_t = \frac{1}{2} ln(\frac{1-\epsilon_t}{\epsilon_t})。这个式子的粗略理解是如果一个样本被误分类,那么下一轮迭代将拥有更大的权值。

Cost-Sensitive Decision Trees

决策树的叶子节点是类别,非叶子节点是规则。建立决策树的过程是从上到下,逐层挑选一个最有分类性的规则,并将样本按照这个规则递归到两个子树处理。

Contact-lens decision tree
决策树

决策树有两个步骤,第一个是从上到下的分裂,第二个是从下到上的修建。

决策树的分裂使用了不纯度函数(impurity function),这个函数是否超过特定阈值决定了一个节点的分裂与否。不同的不纯度函数对于不均衡数据有不同的敏感度,比如传统的准确率函数就非常敏感,而Gini函数、熵函数(Entropy)、DKM函数就不那么敏感。其中,DKM函数生成的未修剪决策树相对较小。

决策树的修剪有利于增加其普遍性,但是对于不均衡数据,决策树的修剪可能会减掉小类的样本,降低模型准确度。类似于Laplace修剪技术也被提出用于让决策树的修剪不那么糟糕。

Cost-Sensitive Neural Networks

一个神经网络,会构造基于模型输出与目标距离的损失函数

(8)   \begin{equation*}E(\omega) = \frac{1}{2} \sum(t_k - o_k)^2\end{equation*}

其中\omega是模型权值,t_k,o_k分别是模型的目标与输出。在每一轮迭代,神经网络都会计算每个权值对于损失函数的梯度,用这个梯度更新权值,使得网络误差函数更小。

(9)   \begin{equation*}\Delta \omega_n = -\eta \nabla_{\omega} E(\omega_n)\end{equation*}

其中\eta是学习速率,\nabla_{\omega}是梯度算子。

成本敏感学习可以以下面四种方式引入模型——

  1. 成本敏感可以体现在对最终估计概率的修正上面
  2. 模型输出o_k可以被修正为成本敏感
  3. 学习速率可以被修正为成本敏感
  4. 误差函数最小化中的误差可以是成本敏感的

这四种方式都各有非常多的研究成果,这里就略去不谈。

基于核函数的算法和主动学习

Kernel-Based Learning Framework

这里讨论支持向量机(SVM)在不均衡数据集下的研究。由于支持向量机试图最小化总误差,因此它们天生就有优化大类样本的偏好。在一个理想模型里面,大类小类通过一条直线分隔,SVM分类的界限将远远偏离实际的界限。

Integration of Kernel Methods with Sampling Methods

大量研究都尝试将采样算法与SVM结合,其中比较有代表性的是GSVM-RU,将GSVM与欠采样算法结合。通过迭代的算法,每一轮迭代,都将大类中被定为支持向量的样本点移出。实现欠采样的目标。

位于最大间隔超平面边界上的点被称为支持向量

Kernel Modification Methods for Imbalanced Learning

直接修改核函数也是很多研究者研究的方向,这些研究的共同点是尝试向原来的SVM添加一个模块或替换一个模块,实现对不平衡数据的支持。包括了LOO-AUC、PSVM、P2PKNNC等等算法

Active Learning Methods for Imbalanced Learning

主动学习通过某种策略找到未进行类别标注的样本数据中最有价值的数据,交由专家进行人工标注后,将标注数据及其类别标签纳入到训练集中迭代优化分类模型,改进模型的处理效果。

在主动学习领域,文章指出,主动学习和基于核函数的学习算法有共通之处,所以有必要放在一起讨论。比如说,一个在SVM算法中靠近超平面的样本点就是比较有意义的。当然,如果遇到了数据不均衡,主动学习也会有很多需要研究的问题。

Additional Methods for Imbalanced Learning

不均衡数据的处理方式还有很多无法被分类到之前类别的算法。比如所有的多标签算法、小样本数量的处理算法、基于单个标签的学习算法等等。

评价指标

使用下表来定义正/负样本被模型正确/错误分类的个数,是后续统计指标的基本依赖变量——

True positives(TP):  被正确地划分为正例的个数,即实际为正例且被分类器划分为正例的实例数;
False positives(FP): 被错误地划分为正例的个数,即实际为负例但被分类器划分为正例的实例数;
False negatives(FN):被错误地划分为负例的个数,即实际为正例但被分类器划分为负例的实例数;
True negatives(TN): 被正确地划分为负例的个数,即实际为负例且被分类器划分为负例的实例数。 

单数值指标

这里假设小类为正类(positive),大类为负类(negative),有准确率(accuracy)和错误率(error rate)

(10)   \begin{equation*}Accuracy = \frac{TP+TN}{P_C+N_C}\end{equation*}

(11)   \begin{equation*}ErrorRate = 1 - Accuracy\end{equation*}

(12)   \begin{equation*}Precesion = \frac{TP}{TP+FP}\end{equation*}

(13)   \begin{equation*}Recall = \frac{TP}{TP+FN}\end{equation*}

(14)   \begin{equation*}F\_{Measure} = \frac{(1+\beta)^2 \cdot Recall \cdot Precesion}{\beta^2 \cdot Recall + Precesion}\end{equation*}

(15)   \begin{equation*}G\_{Mean} = \sqrt{\frac{TP}{TP+FN} \times \frac{TN}{TN+FP}}\end{equation*}

对于(1)(2)来说,虽然比较符合常理,但是容易被欺骗,比如对于5%正类,95%负类的数据,全部分类为负类可以达到95%正确率。这类“对于分布敏感”的指标不是用于不均衡数据集的算法效果评估。

对于精准率(3)来说,描述了“对于所有被分类器预测为正的样本,有多少预测对了?”
对于召回率(4)来说,描述了“有多少真正是正的类别,被正确预测了?”
通过观察公式,可以发现,精准率(3)是数据分布敏感的,而召回率(4)是数据分布不敏感的。
如果使用恰当的话,精准率和召回率已经足以描述用以评价分类器好坏的所有信息了。

F-Measures(5)是精准率率和召回率的合并,并用参数\beta调整双方的权重

G-Mean(6)则使用了正类准确率和负类准确率的比例来刻画分类器的偏好。

上面这些单变量指标虽然已经足够好了,但是还是没法解决一个问题“如何比较两个分类器孰好孰坏?”

ROC曲线

定义真阳性率(TPRate)和假阳性率(FPRate)

(16)   \begin{equation*}TP\_Rate = \frac{TP}{P_C}; FP\_Rate = \frac{FP}{N_C}\end{equation*}

ROC曲线的空间是一个二维平面,横坐标为假阳性率,纵坐标为真阳性率。

对于硬标签分类器(hard-type classifier),输出是类别标签,其效率可以用一个坐标点(FPR, TPR)画在ROC空间上。我们知道完美分类是A(0,1),也就是说越靠近A,分类器的效果越好,而灰色部分则是表示分类器没有随机分类好。

一个软标签分类器(soft-type classifier),输出是类别属于“正类”的概率,这种情况下,我们可以画出一条ROC曲线,在这种情况下,我们可以用曲线下方的面积来衡量分类器的效果。

PR曲线

研究发现ROC曲线有些时候会把模型的结果看的过于乐观,而PR曲线(Precession-Recall Curve)则是为了弥补这一缺憾的。

曲线和ROC曲线的唯一区别在于,ROC曲线使用(FPR, TPR)刻画分类器的效果,而PR曲线则用(Precesion, Recall)来刻画分类器效果。

比较两个曲线的公式,我们会发现FP_Rate在N_C非常大的时候,无法捕捉到FP的数量变化。

代价曲线

代价曲线(Cost Curve)尝试解释的问题是“一个分类器在处理先验概率不同样本表现如何?”。在ROC曲线中的每一个点都对应了代价曲线里面的一条线。

在ROC曲线中的坐标(FP, TP),都可以计算出期望的代价

(17)   \begin{equation*}E(C) = (1-TP-FT)\times PCF(+)+FP\end{equation*}

其中E(C)是期望代价,PCF(+)是样本为正的先验概率。

我们可以将手中若干可用的分类器的代价直线下方区域求交,获得的面积就是代价曲线,该曲线的含义是在“已知先验概率的条件下选择分类器,能够达到的最小期望代价是多少?”

多分类问题的拓展指标

上述的很多指标都可以拓展到多标签分类问题中,包括多标签ROC,M-Measures,Misclassification Cost……

未来我们该怎么做?

了解不均衡分类算法的本质

现在的很多研究都是偏技术的,为某个问题设计了一个巧妙地算法,能够优化某一个评价指标。但是这些算法究竟如何优化了模型表现,并没有人能够说清楚。

  1. 与从原始分布学习相比,什么样的假设会使不平衡学习算法更好地工作?
  2. 应该在多大程度上平衡原始数据集?
  3. 不平衡的数据分布如何影响学习算法的计算复杂度?
  4. 在数据分布不平衡的情况下,普遍的误差范围是多少?
  5. 对于特定的算法和应用领域,是否有一种通用的理论方法可以减轻学习不平衡数据集的障碍?

需要一个标准化的评价体系

上面讲的那些基于曲线的评价指标非常重要,需要行业能够有意识的将其纳入评价指标,这样才能有助于模型的迭代比较。

增量数据上的不均衡学习

在网络监控,流媒体,网络挖掘领域,数据往往是增量到达的,在增量学习如果存在不平衡该怎么办,有以下几个问题需要思考:

  1. 如果在学习期间的中间引入不平衡,我们如何自主调整学习算法?
  2. 我们是否应该考虑在增量学习期间重新平衡数据集?如果是这样,我们怎么能做到这一点呢?
  3. 我们如何积累经验并利用这些知识自适应地改进从新数据中的学习?
  4. 当新引入的概念也不平衡时(即概念漂移的不平衡问题),我们如何处理这种情况?

半监督学习与不均衡数据集

半监督学习中,同时存在带标注和不带标注的数据,这个时候,如果存在不平衡又改如何处理,思考如下问题:

  1. 如何识别未标记的数据示例是来自平衡的还是不平衡的底层分布?
  2. 给定一个带有标签的不平衡训练数据,有哪些有效的方法来恢复未标记的数据示例?
  3. 在给定不平衡标记数据的恢复过程中(通过传统的半监督学习技术),可能会引入什么样的偏差?

参考资料

原始论文:He, H., & Garcia, E. A. (2009). Learning from imbalanced data. IEEE Transactions on knowledge and data engineering21(9), 1263-1284.

机器学习模型性能评估二:代价曲线与性能评估方法总结:https://zhuanlan.zhihu.com/p/53781144

机器学习基础(1)- ROC曲线理解:https://www.jianshu.com/p/2ca96fce7e81

adaboost.M1与adaboost.M2差别比较:https://blog.csdn.net/tyh70537/article/details/76675098

使用Python在音频频谱上绘制文本

这是一段8秒长的音频,采样率22050。总计8 \times 22050 = 176400个采样点可以完美复现这一段时间内,每个采样时刻空气震动的情况。

将这些采样数据输入电脑扬声器,扬声器按照数据指示的强度和方向压缩空气,使空气震动,就可以发出想要的声音。

使用傅里叶变换(DFT),可以找到这个音频在每一个频率处的强度。

什么是傅里叶变换?下面这个视频给出了一个浅显的解释——

由于这段整整8秒的音频并不是一成不变的。所以计算整个音频在每一个频率上的强度并没有太大的意义。可以将整个音频分成若干段,(在本文的例子中,我们将连续的2048个采样点看做一段),分别计算每一段在每一个频率下强度,最终得到一个二维矩阵,这就是短时傅里叶变换(stft)。

这个图像,红色部分意味着这个一时刻的这一个频率的声音强度较大。整张图片可以理解为声音的指纹,在本文中就用声纹称呼它好了。从声纹中,很容易看到原音频的节拍、音色等听觉属性。

在声纹上,可以绘制自己的文字,比如像这样:

由于在声纹文字的绘制只是将某一个时间点,某个频率强度稍微增强,对原音频来说,只是增加了些许噪音,并不会大幅度影响音质。

由此,我们成功将一句话嵌入了音频中。对于外人来看,仅仅是音频多了一些杂音,但是倘若将音频重新进行stft变换,则可以从频谱中读出消息。


安装python依赖库,其中librosa的使用教程可移步《librosa使用教程

pip install librosa
pip install soundfile

引入文字转图片的函数,见我另一篇文章《Python绘制汉字点阵图(字符图)》,将代码放置到charpics.py中。

下面是生成脚本,包含两个函数

  • 函数analyse_audio可以绘制一个音频的dft、stft变换结果,我们可以用它观察一段音频的声纹。
  • 函数modify_audio则可以通过修改声纹,实现音频的修改。
import numpy as np
import librosa
import math
from charpics import gen_pics
import soundfile as sf

def analyse_audio(
        filename,
        duration = None,
        offset = None
        ):
    import matplotlib.pyplot as plt
    import librosa.display as display
    y, sr = librosa.load(filename, duration=duration, offset=offset)
    print("sample rate = %d" % sr)
    M = librosa.feature.melspectrogram(y=y)
    M_highres = librosa.feature.melspectrogram(y=y, hop_length=512)
    F_raw = librosa.core.stft(y=y)
    F_all = np.fft.fft(y)
    plt.figure(figsize=(6, 8))
    ax = plt.subplot(4, 1, 1)
    ax.plot(y)
    ax = plt.subplot(4, 1, 2)
    ax.plot(F_all)
    ax = plt.subplot(4, 1, 3)
    display.specshow(librosa.power_to_db(M, ref=np.max), y_axis='mel', x_axis='time')
    ax = plt.subplot(4, 1, 4)
    display.specshow(librosa.power_to_db(F_raw, ref=np.max), y_axis='linear', x_axis='time')
    plt.show()

def modify_audio(
        text, # the text you wan't to embed into audio
        input_filename,  # input audio file name(mp3/wav)
        output_filename, # output audio file name (wav)
        font_path,  # the location of system font
        offset = 0, # the start position of the audio
        duration = 5, # the duration of the audio
        strength = 3, # strength of the modification
        plot_figure = False # plot the result via matplotlib
        ):
    y, sr = librosa.load(input_filename, duration=duration, offset=offset)
 
    M_raw = librosa.core.stft(y=y)
    M = M_raw.copy()

    _, mask = gen_pics(text, 2, font_path)
    mask = mask.T[::-1,:].astype(np.float32) / 255
    mask_mul = mask*(strength-1/strength) + 1/strength

    ratio = math.ceil(M.shape[1] / mask.shape[1])

    for i in range(M.shape[0]):
        for j in range(M.shape[1]):
            posi = i/M.shape[0]
            posj = j/M.shape[1]
            posi = int(min(posi * mask.shape[0], mask.shape[0]-1))
            posj = int(min(posj * mask.shape[1], mask.shape[1]-1))
            M[i][j] = M[i][j] * mask_mul[posi][posj]

    yy = librosa.core.istft(M)

    if plot_figure:
        import matplotlib.pyplot as plt
        import librosa.display as display
        plt.figure(figsize=(6, 8))
        ax = plt.subplot(2, 1, 1)
        display.specshow(librosa.power_to_db(M_raw, ref=np.max), y_axis='mel', x_axis='time')
        ax = plt.subplot(2, 1, 2)
        display.specshow(librosa.power_to_db(M, ref=np.max), y_axis='mel', x_axis='time')
        plt.show()
    sf.write(output_filename, yy, sr)

if __name__ == '__main__':
    input_filename = './song.mp3'
    output_filename = './song-modified.wav'
    font_path = "/Library/Fonts/Arial Unicode.ttf"

    modify_audio('气氛蕉啄起来', input_filename, output_filename, font_path,
        strength = 2.5, duration = 8, offset = 20, plot_figure=True)

    analyse_audio(output_filename, duration=None, offset=None)

算法的核心就是使用librosa.core.stftlibrosa.core.istft进行变换。在中间,调用gen_pics函数修正声纹,具体来说,让文字部分的声纹强度乘strength,其他部分除以strength

Python绘制汉字点阵图(字符图)

目标:给定一段话,将这段话转换为字符的图片,比如说一句“哈哈哈”可以被转换为下面的字符矩阵。

999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999
999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999
999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999
999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999
999999999999999999   99999999999999999999999999999   99999999999999999999999999999   99999999999
999999999999999999   99999999999999999999999999999   99999999999999999999999999999   99999999999
99        9999999      99999999999        9999999      99999999999        9999999      999999999
99   999  99999   999   9999999999   999  99999   999   9999999999   999  99999   999   99999999
99   999  9999   99999    99999999   999  9999   99999    99999999   999  9999   99999    999999
99   999                    999999   999                    999999   999                    9999
99   999     99999999999999  99999   999     99999999999999  99999   999     99999999999999  999
99   999  999999999999999999999999   999  999999999999999999999999   999  9999999999999999999999
99   999  999             99999999   999  999             99999999   999  999             999999
99        999    999999   99999999        999    999999   99999999        999    999999   999999
99    9   999   99999999  99999999    9   999   99999999  99999999    9   999   99999999  999999
9999999999999   99999999  9999999999999999999   99999999  9999999999999999999   99999999  999999
9999999999999             9999999999999999999             9999999999999999999             999999
9999999999999    999999   9999999999999999999    999999   9999999999999999999    999999   999999
999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999
999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999
999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999
999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999

核心思路:通过python的pygame库,生成文字图片,再逐像素转换为字符图。

安装依赖库

pip install pygame

查找一个中文字体文件,该字体文件将用于后续的字符图生成

locate *.ttf

使用下面的代码进行转换:

#encoding: utf-8
import numpy as np
import os
import pygame
import pygame.surfarray as surfarray
pygame.font.init()


def gen_pics(
    text, # The text you want to generate
    scale, # Larger scale value lead to smaller output figure
    font_path, # A system font path
    white_pixel = '_',
    black_pixel = '+',
    ):
    font = pygame.font.Font(font_path, 64)
    rtext = font.render(text, True, (0, 0, 0), (255, 255, 255))
    img = surfarray.pixels2d(rtext)
    scaleH = scale
    scaleW = max(1, scale // 2)
    result = ''
    for h in range(0,img.shape[1],scaleH):
        for w in range(0,img.shape[0],scaleW):
            grey = img[w:w+scaleH,h:h+scaleW].mean()
            if grey < 128:
                result += black_pixel
            else:
                result += white_pixel
        result += '\n'
    return result, img

if __name__ == '__main__':
    font_path = "/Library/Fonts/Arial Unicode.ttf"
    res, _ = gen_pics(
        text = "哈哈哈",
        scale = 4,
        font_path = font_path,
        white_pixel = ' ',
        black_pixel = '9',
        )
    print(res)

小应用:通过使用相近的字符生成字符图,将信息隐藏的加密方式<字符图加密>

Hyperledger入门学习教程

本文是作者“零基础”入门Hyperledger系列的日记性质的文章。笔者在区块链开发方面零基础,按照文中的顺序学习,三天内能够基本理解Hyperledger系列代码库的功用。


第一步:基础常识(0 hours)

区块链基础

区块链这个概念最近非常火,稍微对这方面有所关注的人都对区块链的核心思路有所了解。在笔者心目中,区块链,一个基于密码学的去中心化系统,该系统拥有不可篡改、匿名等特点。这一层面的理解,读者可以参考知乎上关于区块链的科普文章,在这里就不详细讲解了。

GO语言【可选】

除了比特币是用C++写的之外,大部分的区块链项目都是用GO语言写的,因为GO语言同时具有C++语言的效率,以及Java语言的安全性,广为高性能项目开发者所喜爱。后续的代码阅读需要对GO语言有基础的了解。当然,如果你只需要部署项目的话,Hyperledger已经有足够简单的脚本以致于不需要懂GO语言也能学会。

菜鸟教程:https://www.runoob.com/go/go-tutorial.html
C语言中文网教程:http://c.biancheng.net/golang/

第二步:阅读Hyperledger书籍 (2 hours)

尝试了解:Hyperledger的历史、存在意义以及算法基础。

网络对于Hyperledger的教程文章并不多,而且质量也并不太好,所以想要系统性的搞清楚联盟链有什么东西,可以看有关的书籍。我看的是下面这本书,由于微信阅读的免费卡真的厉害,在kindle上面30+RMB的书在微信阅读都是免费的——

笔者花了一个晚上的时间过了这本书的前几章,觉得重要的知识点有——

  • 区块链的分布式储存技术特性使得其本身就是一个很健壮的分布式数据库。
  • 一些有助于理解区块链原理的算法
    • 哈希算法,包括SHA-384, SHA-512等
    • Merkle树:用于对树形结构数据求得Hash值,高效,支持部分校验
    • 非对称加密:椭圆曲线,RSA加密算法
    • 共识机制:工作量证明(POW),股权证明(POS),委托权益证明(DPOS),拜占庭共识算法(PBFT),投注共识机制(Casper),消逝时间证明(POET)……
  • 智能合约的意义,智能合约本质是一段用某种计算编程语言编写的程序,这段程序可以运行在区块链系统提供的容器中。智能合约代码储存在区块链中,结合区块链提供的防篡改防伪造的特性,智能合约能够和区块链本身非常天然的融合在一起。
  • 区块链,特别是公链,有“性能慢”,“数据不易弹性扩展”,“技术社区不成熟”的缺点。
  • Hyperledger(超级账本)本身是Linux基金发起的推荐区块链技术和交易认证的开源项目。后来已经转变为一个开源的代码库集合,包含9个正式项目和50多个相关模块。其中,笔者最为关注的有
    • Hyperledger-Fabirc:Hyperledger核心项目,一个可灵活配置的分布式共享账本。
    • Hyperledger-Explorer: 一个基于Node.js的区块链配置管理,节点查询的工具。
    • Hyperledger-Burrow:一个智能合同机,采用了部分以太坊虚拟机(EVM)的技术规范。提供了共识引擎,合约引擎的接口。

第三步:理解Hyperledger的相关概念 (2 hours)

Hyperledger官方文档:https://hyperledger-fabric.readthedocs.io/en/master/key_concepts.html
民间总结:https://hackernoon.com/hyperledger-fabric-the-20-most-important-terms-made-simple-2753f925db4

这一步我们需要尝试了解Hyperledger的常用术语,包括Consensus、Policy、Peers、Ledger等概念。

这些概念非常重要,原因是在后续的部署中,我们可能遇到类似于peer chaincode invoke 这样的命令,如果读者连这些名词的具体含义都没有了解的话,在后续的部署中会觉得很难受。

读者可以现在先对这些概念有一个初步的映像,在后续部署过程中遇到的时候,再有针对性的去阅读。

这些概念,可以在Hyperledger Fabric的文档中了解。

重要知识点——

Policy
policy,是一系列规则,描述了决策该如何指定,结果是如何产生的。可以类比于我们平常的“保险协议”,定义了理赔的范围、时间等信息。事实上,hyperledger-fabric的policy和比特币、以太坊等项目的policy有所不同,后者的policy是固定的,过去、现在、未来都不会变。但是hyperledger-fabric则支持网络的所有者,不仅可以在网络运行之前制定规则,还可以在网络运行的时候修改规则。

Peers
peers,对等节点,是网络上的最基本的元素。区块链网络的对等节点内部储存了账单以及智能合约的拷贝。Peer可以被创建、启动、暂停、删除,并且他们暴露了一系列API供管理人员以及服务需求商调用。

Ledger
Ledger 是账单的意思,ledger被储存在peer节点中,通过一系列方式保证了账单的同步。

Chaincode
chaincode,合约代码,和ledger一样储存在peer节点中,而chaincode定义了用来访问ledger的代码。

Orderer
orderer,请求者,可以理解为用户和整个peers网络节点的一个终结。orderer在整个联盟统一的条件下被加入联盟链,拥有权限创建一个交易,并且让所有peers同步这个交易。

第三步:Hyperledger Fabric部署 (4 hours)

首先按照入门教程中的引导,我们可以使用fabric-samples这个库中提供的脚本,在docker环境中跑一个简易的Hyperledger版本。具体来说,我们接下来,将使用fabric-samples/test-network的脚本,一步一步建立起一个网络,这个网络能够记录一系列车辆的信息。

首先我们搭建环境,在工作目录执行

curl -sSL https://bit.ly/2ysbOFE | bash -s

这一个指令干了三件事情——

  • 倘若工作目录下没有fabric-samples,则从github将这个库clone下来
  • 从github的fabric仓库下载可执行文件,放在fabric-samples/bin目录下
  • 从dockerhub上面拉取需要的docker镜像文件

这一步指令可能存在翻墙的问题,如果遇到了的话,可能就需要根据具体情况“挂代理”“改hosts”或“设镜像”。

接下来我们按照测试网络搭建教程中的命令执行指令。这些指令将一步一步建立起整个联盟链,推荐直接去看原文。

进入测试网络的目录

cd fabric-samples/test-network/

开启网络,建立网络中的三个节点

  • orderer.example.com
  • peer0.org1.example.com
  • peer0.org2.example.com

这三个节点每一个都是一个独立的docker容器,其中peer节点一定要依附于某个组织,而orderer节点可以独立存在。命令输出见附录A

./network.sh up

接下来使用命令建立一个channel:

./network.sh createChannel

命令输出见附录B

接下来,使用命令建立一个ChainCode(这里因为要下载GO包,叒需要翻墙了)

./network.sh deployCC

该指令首先安装chaincode的依赖库,然后将chaincode分别安装到两个peer中,接着再将chaincode部署到channel中。具体输出见附录C,可以看到,一些“汽车”的数据被加入了我们的账本——

[{"Key":"CAR0","Record":{"make":"Toyota","model":"Prius","colour":"blue","owner":"Tomoko"}},{"Key":"CAR1","Record":{"make":"Ford","model":"Mustang","colour":"red","owner":"Brad"}},{"Key":"CAR2","Record":{"make":"Hyundai","model":"Tucson","colour":"green","owner":"Jin Soo"}},{"Key":"CAR3","Record":{"make":"Volkswagen","model":"Passat","colour":"yellow","owner":"Max"}},{"Key":"CAR4","Record":{"make":"Tesla","model":"S","colour":"black","owner":"Adriana"}},{"Key":"CAR5","Record":{"make":"Peugeot","model":"205","colour":"purple","owner":"Michel"}},{"Key":"CAR6","Record":{"make":"Chery","model":"S22L","colour":"white","owner":"Aarav"}},{"Key":"CAR7","Record":{"make":"Fiat","model":"Punto","colour":"violet","owner":"Pari"}},{"Key":"CAR8","Record":{"make":"Tata","model":"Nano","colour":"indigo","owner":"Valeria"}},{"Key":"CAR9","Record":{"make":"Holden","model":"Barina","colour":"brown","owner":"Shotaro"}}]

现在,一切初始化操作已经就绪,我们可以开始与我们的网络进行交互。首先确认位于test-network目录,配置环境变量——

export PATH={PWD}/../bin:{PWD}:PATH export FABRIC_CFG_PATH=PWD/../config/
# Environment variables for Org1
export CORE_PEER_TLS_ENABLED=true
export CORE_PEER_LOCALMSPID="Org1MSP"
export CORE_PEER_TLS_ROOTCERT_FILE={PWD}/organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt export CORE_PEER_MSPCONFIGPATH={PWD}/organizations/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp
export CORE_PEER_ADDRESS=localhost:7051

接下来,我们使用一个命令查询账本中的数据:

peer chaincode query -C mychannel -n fabcar -c '{"Args":["queryAllCars"]}'

结果与deployCC的输出结果相同。

使用下面的命令可以对账本进行修改,

peer chaincode invoke -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls true --cafile {PWD}/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem -C mychannel -n fabcar --peerAddresses localhost:7051 --tlsRootCertFiles{PWD}/organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt --peerAddresses localhost:9051 --tlsRootCertFiles ${PWD}/organizations/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt -c '{"function":"changeCarOwner","Args":["CAR9","Dave"]}'

虽然命令的功能可以从字面意思翻译出来,我们还是可以用help命令自习看一看具体每个参数可以怎么用。

peer命令本身可以看做是一个入口命令,里面分为了peer node操作节点,peer channel 操作频道,peer chaincode 操作合约代码。

其中peer chaincode进一步又有installinvoke等子命令。

在之后的第四步中,我们会使用./first-network而非现在的./test-network,所以需要将现在的网络关闭。结果如附录d。

./network.sh down

第四步,部署hyperledger-explorer

项目仓库:https://github.com/hyperledger/blockchain-explorer
非官方安装教程:https://medium.com/coinmonks/hyperledger-explorer-quick-start-50a49c6d7957
官方安装教程:https://github.com/hyperledger/blockchain-explorer#Database-Setup

hyperledger-explorer的部署需要依赖挺多的东西的,单纯安装环境就需要比较长的时间。这一步推荐同时看官方/非官方教程,因为非官方教程有些地方有点过时。

修改nodejs版本

which node
npm install -g n
sudo n 8.11.1
node --version

安装PostgreSQL、jq

brew install postgres
brew install jq

开启Fabric,由于在上一步已经启动了一个Fabric网络,其中临时设置了一些环境变量,所以这里推荐重启终端以消除影响

首先安装GO库

go get github.com/hyperledger/fabric/core/chaincode/shim

接着启动./first-network,这一步需要逐一输出有没有报错,如果有报错的话,不应该继续下去。

cd ./first-network/
./byfn.sh generate
./byfn.sh up

下载blockchain-explorer

git clone https://github.com/hyperledger/blockchain-explorer.git

开启PostgreSQL,并初始化数据库

pg_ctl -D /usr/local/var/postgres start
cd ./blockchain-explorer/app/persistence/fabric/postgreSQL/db
./createdb.sh

查看配置文件vim ./blockchain-explorer/app/platform/fabric/config.json 由于配置文件默认引用了./connection-profile/first-network.json,所以我们需要按照官方文档的指引,稍微修改一下这一部分,包括——

  • 修改配置文件中fabric-path这个字符串为./fabric-samples所在的真实绝对路径。
  • 修改配置文件中以_sk结尾的文件为真实文件名,在我这里是priv_sk

安装npm依赖包

cd ./blockchain-explorer
./main install
./main test

如果安装正确,应该能够通过所有测试。最后执行命令

./start.sh

这个命令没有控制行输出,所有输出被丛定祥到./logs目录下,如果没有成功运行,可以在这个目录的./app/app.log日志里面找原因。如果成功运行,则在http://127.0.0.1:8080/#/这里可以看到浏览器。(这个字号以及右上角的图标栏是真的丑)

附录

A. 命令./network.sh up输出

Starting nodes with CLI timeout of '5' tries and CLI delay of '3' seconds and using database 'leveldb' with crypto from 'cryptogen'

LOCAL_VERSION=2.0.0
DOCKER_IMAGE_VERSION=2.0.0
/Users/toby/Workspace/fabric-samples/test-network/../bin/cryptogen

##########################################################
##### Generate certificates using cryptogen tool #########
##########################################################

##########################################################
############ Create Org1 Identities ######################
##########################################################
+ cryptogen generate --config=./organizations/cryptogen/crypto-config-org1.yaml --output=organizations
org1.example.com
+ res=0
+ set +x
##########################################################
############ Create Org2 Identities ######################
##########################################################
+ cryptogen generate --config=./organizations/cryptogen/crypto-config-org2.yaml --output=organizations
org2.example.com
+ res=0
+ set +x
##########################################################
############ Create Orderer Org Identities ###############
##########################################################
+ cryptogen generate --config=./organizations/cryptogen/crypto-config-orderer.yaml --output=organizations
+ res=0
+ set +x

Generate CCP files for Org1 and Org2
/Users/toby/Workspace/fabric-samples/test-network/../bin/configtxgen
#########  Generating Orderer Genesis block ##############
+ configtxgen -profile TwoOrgsOrdererGenesis -channelID system-channel -outputBlock ./system-genesis-block/genesis.block
2020-03-10 18:43:19.430 CST [common.tools.configtxgen] main -> INFO 001 Loading configuration
2020-03-10 18:43:19.453 CST [common.tools.configtxgen.localconfig] completeInitialization -> INFO 002 orderer type: etcdraft
2020-03-10 18:43:19.454 CST [common.tools.configtxgen.localconfig] completeInitialization -> INFO 003 Orderer.EtcdRaft.Options unset, setting to tick_interval:"500ms" election_tick:10 heartbeat_tick:1 max_inflight_blocks:5 snapshot_interval_size:16777216
2020-03-10 18:43:19.454 CST [common.tools.configtxgen.localconfig] Load -> INFO 004 Loaded configuration: /Users/toby/Workspace/fabric-samples/test-network/configtx/configtx.yaml
2020-03-10 18:43:19.458 CST [common.tools.configtxgen] doOutputBlock -> INFO 005 Generating genesis block
2020-03-10 18:43:19.458 CST [common.tools.configtxgen] doOutputBlock -> INFO 006 Writing genesis block
+ res=0
+ set +x
Creating network "net_test" with the default driver
Creating volume "net_orderer.example.com" with default driver
Creating volume "net_peer0.org1.example.com" with default driver
Creating volume "net_peer0.org2.example.com" with default driver
Creating peer0.org2.example.com ... done
Creating peer0.org1.example.com ... done
Creating orderer.example.com    ... done
CONTAINER ID        IMAGE                               COMMAND                  CREATED             STATUS                     PORTS                              NAMES
f52d0319b985        hyperledger/fabric-orderer:latest   "orderer"                1 second ago        Up Less than a second      0.0.0.0:7050->7050/tcp             orderer.example.com
163555ca9cc3        hyperledger/fabric-peer:latest      "peer node start"        1 second ago        Up Less than a second      0.0.0.0:7051->7051/tcp             peer0.org1.example.com
e73c49a212eb        hyperledger/fabric-peer:latest      "peer node start"        1 second ago        Up Less than a second      7051/tcp, 0.0.0.0:9051->9051/tcp   peer0.org2.example.com

B. 命令./network.sh createChannel输出

Creating channel 'mychannel'.

If network is not up, starting nodes with CLI timeout of '5' tries and CLI delay of '3' seconds and using database 'leveldb

### Generating channel configuration transaction 'mychannel.tx' ###
+ configtxgen -profile TwoOrgsChannel -outputCreateChannelTx ./channel-artifacts/mychannel.tx -channelID mychannel
2020-03-10 20:09:40.336 CST [common.tools.configtxgen] main -> INFO 001 Loading configuration
2020-03-10 20:09:40.358 CST [common.tools.configtxgen.localconfig] Load -> INFO 002 Loaded configuration: /Users/toby/Workspace/fabric-samples/test-network/configtx/configtx.yaml
2020-03-10 20:09:40.358 CST [common.tools.configtxgen] doOutputChannelCreateTx -> INFO 003 Generating new channel configtx
2020-03-10 20:09:40.364 CST [common.tools.configtxgen] doOutputChannelCreateTx -> INFO 004 Writing new channel tx
+ res=0
+ set +x

### Generating channel configuration transaction 'mychannel.tx' ###
#######    Generating anchor peer update for Org1MSP  ##########
+ configtxgen -profile TwoOrgsChannel -outputAnchorPeersUpdate ./channel-artifacts/Org1MSPanchors.tx -channelID mychannel -asOrg Org1MSP
2020-03-10 20:09:40.397 CST [common.tools.configtxgen] main -> INFO 001 Loading configuration
2020-03-10 20:09:40.415 CST [common.tools.configtxgen.localconfig] Load -> INFO 002 Loaded configuration: /Users/toby/Workspace/fabric-samples/test-network/configtx/configtx.yaml
2020-03-10 20:09:40.415 CST [common.tools.configtxgen] doOutputAnchorPeersUpdate -> INFO 003 Generating anchor peer update
2020-03-10 20:09:40.418 CST [common.tools.configtxgen] doOutputAnchorPeersUpdate -> INFO 004 Writing anchor peer update
+ res=0
+ set +x

#######    Generating anchor peer update for Org2MSP  ##########
+ configtxgen -profile TwoOrgsChannel -outputAnchorPeersUpdate ./channel-artifacts/Org2MSPanchors.tx -channelID mychannel -asOrg Org2MSP
2020-03-10 20:09:40.449 CST [common.tools.configtxgen] main -> INFO 001 Loading configuration
2020-03-10 20:09:40.468 CST [common.tools.configtxgen.localconfig] Load -> INFO 002 Loaded configuration: /Users/toby/Workspace/fabric-samples/test-network/configtx/configtx.yaml
2020-03-10 20:09:40.468 CST [common.tools.configtxgen] doOutputAnchorPeersUpdate -> INFO 003 Generating anchor peer update
2020-03-10 20:09:40.470 CST [common.tools.configtxgen] doOutputAnchorPeersUpdate -> INFO 004 Writing anchor peer update
+ res=0
+ set +x

Creating channel mychannel
Using organization 1
+ peer channel create -o localhost:7050 -c mychannel --ordererTLSHostnameOverride orderer.example.com -f ./channel-artifacts/mychannel.tx --outputBlock ./channel-artifacts/mychannel.block --tls true --cafile /Users/toby/Workspace/fabric-samples/test-network/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem
+ res=0
+ set +x
2020-03-10 20:09:43.694 CST [channelCmd] InitCmdFactory -> INFO 001 Endorser and orderer connections initialized
2020-03-10 20:09:43.722 CST [cli.common] readBlock -> INFO 002 Expect block, but got status: &{NOT_FOUND}
2020-03-10 20:09:43.726 CST [channelCmd] InitCmdFactory -> INFO 003 Endorser and orderer connections initialized
2020-03-10 20:09:43.931 CST [cli.common] readBlock -> INFO 004 Expect block, but got status: &{SERVICE_UNAVAILABLE}
2020-03-10 20:09:43.935 CST [channelCmd] InitCmdFactory -> INFO 005 Endorser and orderer connections initialized
2020-03-10 20:09:44.141 CST [cli.common] readBlock -> INFO 006 Expect block, but got status: &{SERVICE_UNAVAILABLE}
2020-03-10 20:09:44.146 CST [channelCmd] InitCmdFactory -> INFO 007 Endorser and orderer connections initialized
2020-03-10 20:09:44.354 CST [cli.common] readBlock -> INFO 008 Expect block, but got status: &{SERVICE_UNAVAILABLE}
2020-03-10 20:09:44.359 CST [channelCmd] InitCmdFactory -> INFO 009 Endorser and orderer connections initialized
2020-03-10 20:09:44.564 CST [cli.common] readBlock -> INFO 00a Expect block, but got status: &{SERVICE_UNAVAILABLE}
2020-03-10 20:09:44.568 CST [channelCmd] InitCmdFactory -> INFO 00b Endorser and orderer connections initialized
2020-03-10 20:09:44.773 CST [cli.common] readBlock -> INFO 00c Received block: 0

===================== Channel 'mychannel' created =====================

Join Org1 peers to the channel...
Using organization 1
+ peer channel join -b ./channel-artifacts/mychannel.block
+ res=0
+ set +x
2020-03-10 20:09:47.847 CST [channelCmd] InitCmdFactory -> INFO 001 Endorser and orderer connections initialized
2020-03-10 20:09:47.873 CST [channelCmd] executeJoin -> INFO 002 Successfully submitted proposal to join channel

Join Org2 peers to the channel...
Using organization 2
+ peer channel join -b ./channel-artifacts/mychannel.block
+ res=0
+ set +x
2020-03-10 20:09:50.943 CST [channelCmd] InitCmdFactory -> INFO 001 Endorser and orderer connections initialized
2020-03-10 20:09:50.970 CST [channelCmd] executeJoin -> INFO 002 Successfully submitted proposal to join channel

Updating anchor peers for org1...
Using organization 1
+ peer channel update -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com -c mychannel -f ./channel-artifacts/Org1MSPanchors.tx --tls true --cafile /Users/toby/Workspace/fabric-samples/test-network/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem
+ res=0
+ set +x
2020-03-10 20:09:51.025 CST [channelCmd] InitCmdFactory -> INFO 001 Endorser and orderer connections initialized
2020-03-10 20:09:51.038 CST [channelCmd] update -> INFO 002 Successfully submitted channel update
===================== Anchor peers updated for org 'Org1MSP' on channel 'mychannel' =====================

Updating anchor peers for org2...
Using organization 2
+ peer channel update -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com -c mychannel -f ./channel-artifacts/Org2MSPanchors.tx --tls true --cafile /Users/toby/Workspace/fabric-samples/test-network/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem
+ res=0
+ set +x
2020-03-10 20:09:54.097 CST [channelCmd] InitCmdFactory -> INFO 001 Endorser and orderer connections initialized
2020-03-10 20:09:54.115 CST [channelCmd] update -> INFO 002 Successfully submitted channel update
===================== Anchor peers updated for org 'Org2MSP' on channel 'mychannel' =====================


========= Channel successfully joined ===========

C. 命令./network.sh createChannel的输出结果

deploying chaincode on channel 'mychannel'

Vendoring Go dependencies ...
~/Workspace/fabric-samples/chaincode/fabcar/go ~/Workspace/fabric-samples/test-network
go: finding github.com/mailru/easyjson v0.0.0-20190614124828-94de47d64c63
go: finding github.com/spf13/pflag v1.0.3
go: finding github.com/spf13/viper v1.3.2
go: finding github.com/inconshreveable/mousetrap v1.0.0
go: finding golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be
go: finding golang.org/x/sys v0.0.0-20190710143415-6ec70d6a5542
go: finding golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a
go: finding golang.org/x/sys v0.0.0-20190616124812-15dcb6c0061f
go: finding golang.org/x/tools v0.0.0-20190614205625-5aca471b1d59
go: finding golang.org/x/tools v0.0.0-20180221164845-07fd8470d635
go: finding golang.org/x/text v0.3.2
go: finding github.com/cpuguy83/go-md2man v1.0.10
go: finding golang.org/x/crypto v0.0.0-20190611184440-5c40567a22f8
go: finding github.com/mitchellh/go-homedir v1.1.0
go: finding github.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6
go: finding golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e
go: finding github.com/hashicorp/hcl v1.0.0
go: finding github.com/spf13/cast v1.3.0
go: finding github.com/spf13/jwalterweatherman v1.0.0
go: finding golang.org/x/sys v0.0.0-20181205085412-a5c9d58dba9a
go: finding golang.org/x/text v0.3.0
go: finding github.com/russross/blackfriday v1.5.2
go: finding github.com/spf13/afero v1.1.2
go: finding github.com/fsnotify/fsnotify v1.4.7
go: finding github.com/magiconair/properties v1.8.0
go: finding github.com/coreos/go-semver v0.2.0
go: finding github.com/pelletier/go-toml v1.2.0
go: finding golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2
go: finding github.com/coreos/etcd v3.3.10+incompatible
go: finding golang.org/x/tools v0.0.0-20190311212946-11955173bddd
go: finding github.com/mitchellh/mapstructure v1.1.2
go: finding gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405
go: finding github.com/coreos/go-etcd v2.0.0+incompatible
go: finding github.com/stretchr/testify v1.2.2
go: finding github.com/ugorji/go/codec v0.0.0-20181204163529-d75b2dcb6bc8
go: finding github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77
go: finding golang.org/x/crypto v0.0.0-20181203042331-505ab145d0a9
go: finding github.com/kr/pty v1.1.1
go: finding golang.org/x/sys v0.0.0-20190515120540-06a5c4944438
go: finding golang.org/x/sys v0.0.0-20190412213103-97732733099d
go: finding golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3
go: finding github.com/stretchr/objx v0.1.1
go: finding github.com/konsorten/go-windows-terminal-sequences v1.0.1
go: finding golang.org/x/crypto v0.0.0-20190621222207-cc06ce4a13d4
go: finding github.com/go-openapi/swag v0.19.2
go: finding golang.org/x/tools v0.0.0-20190624180213-70d37148ca0c
go: finding golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135
go: finding golang.org/x/sys v0.0.0-20190422165155-953cdadca894
go: downloading github.com/hyperledger/fabric-contract-api-go v1.0.0
go: downloading github.com/xeipuuv/gojsonschema v1.2.0
go: downloading github.com/hyperledger/fabric-protos-go v0.0.0-20200124220212-e9cfc186ba7b
go: downloading github.com/hyperledger/fabric-chaincode-go v0.0.0-20200128192331-2d899240a7ed
go: downloading github.com/go-openapi/spec v0.19.4
go: downloading github.com/gobuffalo/packr v1.30.1
go: downloading github.com/golang/protobuf v1.3.2
go: downloading google.golang.org/grpc v1.23.0
go: downloading github.com/gobuffalo/envy v1.7.0
go: downloading golang.org/x/crypto v0.0.0-20190621222207-cc06ce4a13d4
go: downloading github.com/gobuffalo/packd v0.3.0
go: downloading github.com/joho/godotenv v1.3.0
go: downloading github.com/go-openapi/jsonreference v0.19.2
go: downloading github.com/go-openapi/swag v0.19.5
go: downloading github.com/rogpeppe/go-internal v1.3.0
go: downloading github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415
go: downloading github.com/mailru/easyjson v0.0.0-20190626092158-b2ccc519800e
go: downloading github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f
go: downloading golang.org/x/net v0.0.0-20190827160401-ba9fcec4b297
go: downloading gopkg.in/yaml.v2 v2.2.2
go: downloading github.com/go-openapi/jsonpointer v0.19.3
go: downloading golang.org/x/sys v0.0.0-20190710143415-6ec70d6a5542
go: downloading github.com/PuerkitoBio/purell v1.1.1
go: downloading google.golang.org/genproto v0.0.0-20180831171423-11092d34479b
go: downloading golang.org/x/text v0.3.2
go: downloading github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578
~/Workspace/fabric-samples/test-network
Finished vendoring Go dependencies
Using organization 1
++ peer lifecycle chaincode package fabcar.tar.gz --path ../chaincode/fabcar/go/ --lang golang --label fabcar_1
++ res=0
++ set +x
===================== Chaincode is packaged on peer0.org1 =====================

Installing chaincode on peer0.org1...
Using organization 1
++ peer lifecycle chaincode install fabcar.tar.gz
++ res=0
++ set +x
2020-03-10 20:45:03.834 CST [cli.lifecycle.chaincode] submitInstallProposal -> INFO 001 Installed remotely: response:<status:200 payload:"\nIfabcar_1:26fb56d416d999e58bd47a1381717065c66abad574736ae11518073fb8c2a747\022\010fabcar_1" >
2020-03-10 20:45:03.835 CST [cli.lifecycle.chaincode] submitInstallProposal -> INFO 002 Chaincode code package identifier: fabcar_1:26fb56d416d999e58bd47a1381717065c66abad574736ae11518073fb8c2a747
===================== Chaincode is installed on peer0.org1 =====================

Install chaincode on peer0.org2...
Using organization 2
++ peer lifecycle chaincode install fabcar.tar.gz
++ res=0
++ set +x
2020-03-10 20:45:19.446 CST [cli.lifecycle.chaincode] submitInstallProposal -> INFO 001 Installed remotely: response:<status:200 payload:"\nIfabcar_1:26fb56d416d999e58bd47a1381717065c66abad574736ae11518073fb8c2a747\022\010fabcar_1" >
2020-03-10 20:45:19.446 CST [cli.lifecycle.chaincode] submitInstallProposal -> INFO 002 Chaincode code package identifier: fabcar_1:26fb56d416d999e58bd47a1381717065c66abad574736ae11518073fb8c2a747
===================== Chaincode is installed on peer0.org2 =====================

Using organization 1
++ peer lifecycle chaincode queryinstalled
++ res=0
++ set +x
Installed chaincodes on peer:
Package ID: fabcar_1:26fb56d416d999e58bd47a1381717065c66abad574736ae11518073fb8c2a747, Label: fabcar_1
PackageID is fabcar_1:26fb56d416d999e58bd47a1381717065c66abad574736ae11518073fb8c2a747
===================== Query installed successful on peer0.org1 on channel =====================

Using organization 1
++ peer lifecycle chaincode approveformyorg -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls true --cafile /Users/toby/Workspace/fabric-samples/test-network/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem --channelID mychannel --name fabcar --version 1 --init-required --package-id fabcar_1:26fb56d416d999e58bd47a1381717065c66abad574736ae11518073fb8c2a747 --sequence 1
++ set +x
2020-03-10 20:45:21.675 CST [chaincodeCmd] ClientWait -> INFO 001 txid [45755d771fb952c6a9d251c8be3a011edabc50b861bb82b3b8bb0f9a31c3ad6a] committed with status (VALID) at
===================== Chaincode definition approved on peer0.org1 on channel 'mychannel' =====================

Using organization 1
===================== Checking the commit readiness of the chaincode definition on peer0.org1 on channel 'mychannel'... =====================
Attempting to check the commit readiness of the chaincode definition on peer0.org1 secs
++ peer lifecycle chaincode checkcommitreadiness --channelID mychannel --name fabcar --version 1 --sequence 1 --output json --init-required
++ res=0
++ set +x
{
	"approvals": {
		"Org1MSP": true,
		"Org2MSP": false
	}
}
===================== Checking the commit readiness of the chaincode definition successful on peer0.org1 on channel 'mychannel' =====================
Using organization 2
===================== Checking the commit readiness of the chaincode definition on peer0.org2 on channel 'mychannel'... =====================
Attempting to check the commit readiness of the chaincode definition on peer0.org2 secs
++ peer lifecycle chaincode checkcommitreadiness --channelID mychannel --name fabcar --version 1 --sequence 1 --output json --init-required
++ res=0
++ set +x
{
	"approvals": {
		"Org1MSP": true,
		"Org2MSP": false
	}
}
===================== Checking the commit readiness of the chaincode definition successful on peer0.org2 on channel 'mychannel' =====================
Using organization 2
++ peer lifecycle chaincode approveformyorg -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls true --cafile /Users/toby/Workspace/fabric-samples/test-network/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem --channelID mychannel --name fabcar --version 1 --init-required --package-id fabcar_1:26fb56d416d999e58bd47a1381717065c66abad574736ae11518073fb8c2a747 --sequence 1
++ set +x
2020-03-10 20:45:29.955 CST [chaincodeCmd] ClientWait -> INFO 001 txid [d102dff36c7e097155b0209f8e211d2ca9c0c8d487ea7d3fc837d13b9d48c19e] committed with status (VALID) at
===================== Chaincode definition approved on peer0.org2 on channel 'mychannel' =====================

Using organization 1
===================== Checking the commit readiness of the chaincode definition on peer0.org1 on channel 'mychannel'... =====================
Attempting to check the commit readiness of the chaincode definition on peer0.org1 secs
++ peer lifecycle chaincode checkcommitreadiness --channelID mychannel --name fabcar --version 1 --sequence 1 --output json --init-required
++ res=0
++ set +x
{
	"approvals": {
		"Org1MSP": true,
		"Org2MSP": true
	}
}
===================== Checking the commit readiness of the chaincode definition successful on peer0.org1 on channel 'mychannel' =====================
Using organization 2
===================== Checking the commit readiness of the chaincode definition on peer0.org2 on channel 'mychannel'... =====================
Attempting to check the commit readiness of the chaincode definition on peer0.org2 secs
++ peer lifecycle chaincode checkcommitreadiness --channelID mychannel --name fabcar --version 1 --sequence 1 --output json --init-required
++ res=0
++ set +x
{
	"approvals": {
		"Org1MSP": true,
		"Org2MSP": true
	}
}
===================== Checking the commit readiness of the chaincode definition successful on peer0.org2 on channel 'mychannel' =====================
Using organization 1
Using organization 2
++ peer lifecycle chaincode commit -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls true --cafile /Users/toby/Workspace/fabric-samples/test-network/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem --channelID mychannel --name fabcar --peerAddresses localhost:7051 --tlsRootCertFiles /Users/toby/Workspace/fabric-samples/test-network/organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt --peerAddresses localhost:9051 --tlsRootCertFiles /Users/toby/Workspace/fabric-samples/test-network/organizations/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt --version 1 --sequence 1 --init-required
++ res=0
++ set +x
2020-03-10 20:45:38.425 CST [chaincodeCmd] ClientWait -> INFO 001 txid [6315e52c4b5e435f53b1d10fe6c68d796b8859a93a9bceb27a68179e35a01979] committed with status (VALID) at localhost:7051
2020-03-10 20:45:38.433 CST [chaincodeCmd] ClientWait -> INFO 002 txid [6315e52c4b5e435f53b1d10fe6c68d796b8859a93a9bceb27a68179e35a01979] committed with status (VALID) at localhost:9051
===================== Chaincode definition committed on channel 'mychannel' =====================

Using organization 1
===================== Querying chaincode definition on peer0.org1 on channel 'mychannel'... =====================
Attempting to Query committed status on peer0.org1, Retry after 3 seconds.
++ peer lifecycle chaincode querycommitted --channelID mychannel --name fabcar
++ res=0
++ set +x

Committed chaincode definition for chaincode 'fabcar' on channel 'mychannel':
Version: 1, Sequence: 1, Endorsement Plugin: escc, Validation Plugin: vscc, Approvals: [Org1MSP: true, Org2MSP: true]
===================== Query chaincode definition successful on peer0.org1 on channel 'mychannel' =====================

Using organization 2
===================== Querying chaincode definition on peer0.org2 on channel 'mychannel'... =====================
Attempting to Query committed status on peer0.org2, Retry after 3 seconds.
++ peer lifecycle chaincode querycommitted --channelID mychannel --name fabcar
++ res=0
++ set +x

Committed chaincode definition for chaincode 'fabcar' on channel 'mychannel':
Version: 1, Sequence: 1, Endorsement Plugin: escc, Validation Plugin: vscc, Approvals: [Org1MSP: true, Org2MSP: true]
===================== Query chaincode definition successful on peer0.org2 on channel 'mychannel' =====================

Using organization 1
Using organization 2
++ peer chaincode invoke -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls true --cafile /Users/toby/Workspace/fabric-samples/test-network/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem -C mychannel -n fabcar --peerAddresses localhost:7051 --tlsRootCertFiles /Users/toby/Workspace/fabric-samples/test-network/organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt --peerAddresses localhost:9051 --tlsRootCertFiles /Users/toby/Workspace/fabric-samples/test-network/organizations/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt --isInit -c '{"function":"initLedger","Args":[]}'
++ res=0
++ set +x
2020-03-10 20:45:44.696 CST [chaincodeCmd] chaincodeInvokeOrQuery -> INFO 001 Chaincode invoke successful. result: status:200
===================== Invoke transaction successful on peer0.org1 peer0.org2 on channel 'mychannel' =====================

Using organization 1
Using organization 2
++ peer chaincode invoke -o localhost:7050 --ordererTLSHostnameOverride orderer.example.com --tls true --cafile /Users/toby/Workspace/fabric-samples/test-network/organizations/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem -C mychannel -n fabcar --peerAddresses localhost:7051 --tlsRootCertFiles /Users/toby/Workspace/fabric-samples/test-network/organizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt --peerAddresses localhost:9051 --tlsRootCertFiles /Users/toby/Workspace/fabric-samples/test-network/organizations/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt -c '{"function":"initLedger","Args":[]}'
++ res=0
++ set +x
2020-03-10 20:45:54.795 CST [chaincodeCmd] chaincodeInvokeOrQuery -> INFO 001 Chaincode invoke successful. result: status:200
===================== Invoke transaction successful on peer0.org1 peer0.org2 on channel 'mychannel' =====================

Querying chaincode on peer0.org1...
Using organization 1
===================== Querying on peer0.org1 on channel 'mychannel'... =====================
Attempting to Query peer0.org1 ...1583844357 secs
++ peer chaincode query -C mychannel -n fabcar -c '{"Args":["queryAllCars"]}'
++ res=0
++ set +x

[{"Key":"CAR0","Record":{"make":"Toyota","model":"Prius","colour":"blue","owner":"Tomoko"}},{"Key":"CAR1","Record":{"make":"Ford","model":"Mustang","colour":"red","owner":"Brad"}},{"Key":"CAR2","Record":{"make":"Hyundai","model":"Tucson","colour":"green","owner":"Jin Soo"}},{"Key":"CAR3","Record":{"make":"Volkswagen","model":"Passat","colour":"yellow","owner":"Max"}},{"Key":"CAR4","Record":{"make":"Tesla","model":"S","colour":"black","owner":"Adriana"}},{"Key":"CAR5","Record":{"make":"Peugeot","model":"205","colour":"purple","owner":"Michel"}},{"Key":"CAR6","Record":{"make":"Chery","model":"S22L","colour":"white","owner":"Aarav"}},{"Key":"CAR7","Record":{"make":"Fiat","model":"Punto","colour":"violet","owner":"Pari"}},{"Key":"CAR8","Record":{"make":"Tata","model":"Nano","colour":"indigo","owner":"Valeria"}},{"Key":"CAR9","Record":{"make":"Holden","model":"Barina","colour":"brown","owner":"Shotaro"}}]
===================== Query successful on peer0.org1 on channel 'mychannel' =====================

D. 命令./network.sh down 运行结果

Stopping network

Stopping orderer.example.com    ... done
Stopping peer0.org1.example.com ... done
Stopping peer0.org2.example.com ... done
Removing orderer.example.com    ... done
Removing peer0.org1.example.com ... done
Removing peer0.org2.example.com ... done
Removing network net_default
WARNING: Network net_default not found.
Removing network net_test
Removing volume net_orderer.example.com
Removing volume net_peer0.org1.example.com
Removing volume net_peer0.org2.example.com
Removing network net_test
WARNING: Network net_test not found.
Removing volume net_peer0.org3.example.com
WARNING: Volume net_peer0.org3.example.com not found.
9e8c15a64cbe
a4072998f502
Untagged: dev-peer0.org2.example.com-fabcar_1-26fb56d416d999e58bd47a1381717065c66abad574736ae11518073fb8c2a747-26cde13f78aa8eff73d4744d561a8b137c9eda176ba8a84d36af48a8c9de631c:latest
Deleted: sha256:dc700dd2db09006dd2fce3342fae1216b25e9d38f041b6a0939cebd7028926a8
Deleted: sha256:c407d2e27d1274d907f5951233dd4be58a316d9b044e2ab0c11d760a83de2e95
Deleted: sha256:01895489c6d960088a38bc3900a9f95304033265bcdaed8c4d3a98b8de94b49c
Deleted: sha256:ba1afd3832d77b10bec39ffda37b1db034d74b0564c8ebb70d857bc6b55a1338
Untagged: dev-peer0.org1.example.com-fabcar_1-26fb56d416d999e58bd47a1381717065c66abad574736ae11518073fb8c2a747-61a1af104025983f31a73f6b597bc3a2bf3bc5f680287147657ca79e648bb813:latest
Deleted: sha256:28b9dda318f39b007764e52b6be09ddfea61a208dc4c0baa2963b3f76564c0e6
Deleted: sha256:6bf70e8ade3bf7e98468a901e146f32b8c550b5ba75470cecccaa88030144b74
Deleted: sha256:329afa3cc5d41c17306a013014f3ca7621f7469e969ce500826426414b1d98b1
Deleted: sha256:24d8a9329711871ebe3ac33cdccfa6453c1d584313f0875809529af348301933

First-Network的./byfn.sh up运行结果

./byfn.sh up
Starting for channel 'mychannel' with CLI timeout of '10' seconds and CLI delay of '3' seconds
Continue? [Y/n] y
proceeding ...
LOCAL_VERSION=2.0.0
DOCKER_IMAGE_VERSION=2.0.0
Creating network "net_byfn" with the default driver
Creating volume "net_orderer.example.com" with default driver
Creating volume "net_peer0.org1.example.com" with default driver
Creating volume "net_peer1.org1.example.com" with default driver
Creating volume "net_peer0.org2.example.com" with default driver
Creating volume "net_peer1.org2.example.com" with default driver
Creating volume "net_orderer2.example.com" with default driver
Creating volume "net_orderer3.example.com" with default driver
Creating volume "net_orderer4.example.com" with default driver
Creating volume "net_orderer5.example.com" with default driver
Creating peer1.org1.example.com ... done
Creating peer1.org2.example.com ... done
Creating peer0.org1.example.com ... done
Creating orderer2.example.com   ... done
Creating orderer3.example.com   ... done
Creating orderer.example.com    ... done
Creating orderer4.example.com   ... done
Creating peer0.org2.example.com ... done
Creating orderer5.example.com   ... done
Creating cli                    ... done
CONTAINER ID        IMAGE                               COMMAND             CREATED             STATUS                  PORTS                                NAMES
e32aad7014ed        hyperledger/fabric-tools:latest     "/bin/bash"         1 second ago        Up Less than a second                                        cli
1249b849b0a9        hyperledger/fabric-orderer:latest   "orderer"           4 seconds ago       Up Less than a second   7050/tcp, 0.0.0.0:10050->10050/tcp   orderer4.example.com
6a6f47c0ee97        hyperledger/fabric-orderer:latest   "orderer"           4 seconds ago       Up Less than a second   7050/tcp, 0.0.0.0:8050->8050/tcp     orderer2.example.com
4220ce4e0ea7        hyperledger/fabric-peer:latest      "peer node start"   4 seconds ago       Up 1 second             7051/tcp, 0.0.0.0:9051->9051/tcp     peer0.org2.example.com
3be50ae18707        hyperledger/fabric-orderer:latest   "orderer"           4 seconds ago       Up 1 second             7050/tcp, 0.0.0.0:11050->11050/tcp   orderer5.example.com
b8e736266e0c        hyperledger/fabric-peer:latest      "peer node start"   4 seconds ago       Up 2 seconds            7051/tcp, 0.0.0.0:10051->10051/tcp   peer1.org2.example.com
597566c99ea5        hyperledger/fabric-orderer:latest   "orderer"           4 seconds ago       Up 2 seconds            7050/tcp, 0.0.0.0:9050->9050/tcp     orderer3.example.com
04dd5781d0bd        hyperledger/fabric-orderer:latest   "orderer"           4 seconds ago       Up 2 seconds            0.0.0.0:7050->7050/tcp               orderer.example.com
53a140d598be        hyperledger/fabric-peer:latest      "peer node start"   4 seconds ago       Up 2 seconds            0.0.0.0:7051->7051/tcp               peer0.org1.example.com
c6f4104d41b7        hyperledger/fabric-peer:latest      "peer node start"   4 seconds ago       Up 2 seconds            7051/tcp, 0.0.0.0:8051->8051/tcp     peer1.org1.example.com
Sleeping 15s to allow Raft cluster to complete booting
Vendoring Go dependencies ...
~/Workspace/fabric-samples/chaincode/abstore/go ~/Workspace/fabric-samples/first-network
go: finding golang.org/x/net v0.0.0-20190813141303-74dc4d7220e7
go: finding golang.org/x/sys v0.0.0-20190813064441-fde4db37ae7a
go: finding google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55
go: finding golang.org/x/tools v0.0.0-20190226205152-f727befe758c
go: finding golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961
go: finding golang.org/x/exp v0.0.0-20190121172915-509febef88a4
go: finding golang.org/x/net v0.0.0-20190213061140-3a22650c66bd
go: finding google.golang.org/grpc v1.19.0
go: finding golang.org/x/sync v0.0.0-20181108010431-42b317875d0f
go: finding google.golang.org/appengine v1.4.0
go: finding honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099
go: finding golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3
go: finding golang.org/x/tools v0.0.0-20190114222345-bf090417da8b
go: finding golang.org/x/net v0.0.0-20180826012351-8a410e7b638d
go: finding golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f
go: finding golang.org/x/sys v0.0.0-20180830151530-49385e6e1522
go: finding golang.org/x/net v0.0.0-20180724234803-3673e40ba225
go: downloading github.com/hyperledger/fabric-protos-go v0.0.0-20190821214336-621b908d5022
go: downloading github.com/hyperledger/fabric-chaincode-go v0.0.0-20190823162523-04390e015b85
go: downloading golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2
go: downloading golang.org/x/net v0.0.0-20190813141303-74dc4d7220e7
go: downloading google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55
go: downloading golang.org/x/sys v0.0.0-20190813064441-fde4db37ae7a
~/Workspace/fabric-samples/first-network
Finished vendoring Go dependencies

 ____    _____      _      ____    _____
/ ___|  |_   _|    / \    |  _ \  |_   _|
\___ \    | |     / _ \   | |_) |   | |
 ___) |   | |    / ___ \  |  _ <    | |
|____/    |_|   /_/   \_\ |_| \_\   |_|

Build your first network (BYFN) end-to-end test

Channel name : mychannel
Creating channel...
+ peer channel create -o orderer.example.com:7050 -c mychannel -f ./channel-artifacts/channel.tx --tls true --cafile /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem
+ res=0
+ set +x
2020-03-18 08:32:09.299 UTC [channelCmd] InitCmdFactory -> INFO 001 Endorser and orderer connections initialized
2020-03-18 08:32:09.416 UTC [cli.common] readBlock -> INFO 002 Expect block, but got status: &{NOT_FOUND}
2020-03-18 08:32:09.477 UTC [channelCmd] InitCmdFactory -> INFO 003 Endorser and orderer connections initialized
2020-03-18 08:32:09.680 UTC [cli.common] readBlock -> INFO 004 Expect block, but got status: &{SERVICE_UNAVAILABLE}
2020-03-18 08:32:09.711 UTC [channelCmd] InitCmdFactory -> INFO 005 Endorser and orderer connections initialized
2020-03-18 08:32:09.914 UTC [cli.common] readBlock -> INFO 006 Expect block, but got status: &{SERVICE_UNAVAILABLE}
2020-03-18 08:32:09.931 UTC [channelCmd] InitCmdFactory -> INFO 007 Endorser and orderer connections initialized
2020-03-18 08:32:10.134 UTC [cli.common] readBlock -> INFO 008 Expect block, but got status: &{SERVICE_UNAVAILABLE}
2020-03-18 08:32:10.156 UTC [channelCmd] InitCmdFactory -> INFO 009 Endorser and orderer connections initialized
2020-03-18 08:32:10.358 UTC [cli.common] readBlock -> INFO 00a Expect block, but got status: &{SERVICE_UNAVAILABLE}
2020-03-18 08:32:10.377 UTC [channelCmd] InitCmdFactory -> INFO 00b Endorser and orderer connections initialized
2020-03-18 08:32:10.583 UTC [cli.common] readBlock -> INFO 00c Received block: 0
===================== Channel 'mychannel' created =====================

Having all peers join the channel...
+ peer channel join -b mychannel.block
+ res=0
+ set +x
2020-03-18 08:32:10.827 UTC [channelCmd] InitCmdFactory -> INFO 001 Endorser and orderer connections initialized
2020-03-18 08:32:10.918 UTC [channelCmd] executeJoin -> INFO 002 Successfully submitted proposal to join channel
===================== peer0.org1 joined channel 'mychannel' =====================

+ peer channel join -b mychannel.block
+ res=0
+ set +x
2020-03-18 08:32:14.160 UTC [channelCmd] InitCmdFactory -> INFO 001 Endorser and orderer connections initialized
2020-03-18 08:32:14.247 UTC [channelCmd] executeJoin -> INFO 002 Successfully submitted proposal to join channel
===================== peer1.org1 joined channel 'mychannel' =====================
+ peer channel join -b mychannel.block

+ res=0
+ set +x
2020-03-18 08:32:17.436 UTC [channelCmd] InitCmdFactory -> INFO 001 Endorser and orderer connections initialized
2020-03-18 08:32:17.482 UTC [channelCmd] executeJoin -> INFO 002 Successfully submitted proposal to join channel
===================== peer0.org2 joined channel 'mychannel' =====================

+ peer channel join -b mychannel.block
+ res=0
+ set +x
2020-03-18 08:32:20.665 UTC [channelCmd] InitCmdFactory -> INFO 001 Endorser and orderer connections initialized
2020-03-18 08:32:20.721 UTC [channelCmd] executeJoin -> INFO 002 Successfully submitted proposal to join channel
===================== peer1.org2 joined channel 'mychannel' =====================

Updating anchor peers for org1...
+ peer channel update -o orderer.example.com:7050 -c mychannel -f ./channel-artifacts/Org1MSPanchors.tx --tls true --cafile /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem
+ res=0
+ set +x
2020-03-18 08:32:23.897 UTC [channelCmd] InitCmdFactory -> INFO 001 Endorser and orderer connections initialized
2020-03-18 08:32:23.940 UTC [channelCmd] update -> INFO 002 Successfully submitted channel update
===================== Anchor peers updated for org 'Org1MSP' on channel 'mychannel' =====================

Updating anchor peers for org2...
+ peer channel update -o orderer.example.com:7050 -c mychannel -f ./channel-artifacts/Org2MSPanchors.tx --tls true --cafile /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem
+ res=0
+ set +x
2020-03-18 08:32:27.117 UTC [channelCmd] InitCmdFactory -> INFO 001 Endorser and orderer connections initialized
2020-03-18 08:32:27.159 UTC [channelCmd] update -> INFO 002 Successfully submitted channel update
===================== Anchor peers updated for org 'Org2MSP' on channel 'mychannel' =====================

+ peer lifecycle chaincode package mycc.tar.gz --path github.com/hyperledger/fabric-samples/chaincode/abstore/go/ --lang golang --label mycc_1
+ res=0
+ set +x
===================== Chaincode is packaged on peer0.org1 =====================

Installing chaincode on peer0.org1...
+ peer lifecycle chaincode install mycc.tar.gz
+ res=0
+ set +x
2020-03-18 08:33:04.684 UTC [cli.lifecycle.chaincode] submitInstallProposal -> INFO 001 Installed remotely: response:<status:200 payload:"\nGmycc_1:2d8de5706ec819943d4a98931bbb377a66b669092ca559a61d0e6721e38af7ca\022\006mycc_1" >
2020-03-18 08:33:04.685 UTC [cli.lifecycle.chaincode] submitInstallProposal -> INFO 002 Chaincode code package identifier: mycc_1:2d8de5706ec819943d4a98931bbb377a66b669092ca559a61d0e6721e38af7ca
===================== Chaincode is installed on peer0.org1 =====================

Install chaincode on peer0.org2...
+ peer lifecycle chaincode install mycc.tar.gz
+ res=0
+ set +x
2020-03-18 08:33:27.125 UTC [cli.lifecycle.chaincode] submitInstallProposal -> INFO 001 Installed remotely: response:<status:200 payload:"\nGmycc_1:2d8de5706ec819943d4a98931bbb377a66b669092ca559a61d0e6721e38af7ca\022\006mycc_1" >
2020-03-18 08:33:27.125 UTC [cli.lifecycle.chaincode] submitInstallProposal -> INFO 002 Chaincode code package identifier: mycc_1:2d8de5706ec819943d4a98931bbb377a66b669092ca559a61d0e6721e38af7ca
===================== Chaincode is installed on peer0.org2 =====================

+ peer lifecycle chaincode queryinstalled
+ res=0
+ set +x
Installed chaincodes on peer:
Package ID: mycc_1:2d8de5706ec819943d4a98931bbb377a66b669092ca559a61d0e6721e38af7ca, Label: mycc_1
PackageID is mycc_1:2d8de5706ec819943d4a98931bbb377a66b669092ca559a61d0e6721e38af7ca
===================== Query installed successful on peer0.org1 on channel =====================

+ peer lifecycle chaincode approveformyorg --tls true --cafile /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem --channelID mychannel --name mycc --version 1 --init-required --package-id mycc_1:2d8de5706ec819943d4a98931bbb377a66b669092ca559a61d0e6721e38af7ca --sequence 1 --waitForEvent
+ set +x
2020-03-18 08:33:27.531 UTC [cli.lifecycle.chaincode] setOrdererClient -> INFO 001 Retrieved channel (mychannel) orderer endpoint: orderer.example.com:7050
2020-03-18 08:33:29.681 UTC [chaincodeCmd] ClientWait -> INFO 002 txid [9224f6b805c828dd052160b3441ec90c9c5990c2819a280a564672a425db9c18] committed with status (VALID) at
===================== Chaincode definition approved on peer0.org1 on channel 'mychannel' =====================

===================== Checking the commit readiness of the chaincode definition on peer0.org1 on channel 'mychannel'... =====================
+ peer lifecycle chaincode checkcommitreadiness --channelID mychannel --name mycc --version 1 --sequence 1 --output json --init-required
Attempting to check the commit readiness of the chaincode definition on peer0.org1 ...3 secs
+ res=0
+ set +x

{
	"approvals": {
		"Org1MSP": true,
		"Org2MSP": false
	}
}
===================== Checking the commit readiness of the chaincode definition successful on peer0.org1 on channel 'mychannel' =====================
===================== Checking the commit readiness of the chaincode definition on peer0.org2 on channel 'mychannel'... =====================
+ peer lifecycle chaincode checkcommitreadiness --channelID mychannel --name mycc --version 1 --sequence 1 --output json --init-required
Attempting to check the commit readiness of the chaincode definition on peer0.org2 ...3 secs
+ res=0
+ set +x

{
	"approvals": {
		"Org1MSP": true,
		"Org2MSP": false
	}
}
===================== Checking the commit readiness of the chaincode definition successful on peer0.org2 on channel 'mychannel' =====================
+ peer lifecycle chaincode approveformyorg --tls true --cafile /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem --channelID mychannel --name mycc --version 1 --init-required --package-id mycc_1:2d8de5706ec819943d4a98931bbb377a66b669092ca559a61d0e6721e38af7ca --sequence 1 --waitForEvent
+ set +x
2020-03-18 08:33:36.191 UTC [cli.lifecycle.chaincode] setOrdererClient -> INFO 001 Retrieved channel (mychannel) orderer endpoint: orderer.example.com:7050
2020-03-18 08:33:38.272 UTC [chaincodeCmd] ClientWait -> INFO 002 txid [4997f2f30c647c7362c896e3d88f856e33020ad9229233b138842e6e3506269f] committed with status (VALID) at
===================== Chaincode definition approved on peer0.org2 on channel 'mychannel' =====================

===================== Checking the commit readiness of the chaincode definition on peer0.org1 on channel 'mychannel'... =====================
+ peer lifecycle chaincode checkcommitreadiness --channelID mychannel --name mycc --version 1 --sequence 1 --output json --init-required
Attempting to check the commit readiness of the chaincode definition on peer0.org1 ...3 secs
+ res=0
+ set +x

{
	"approvals": {
		"Org1MSP": true,
		"Org2MSP": true
	}
}
===================== Checking the commit readiness of the chaincode definition successful on peer0.org1 on channel 'mychannel' =====================
===================== Checking the commit readiness of the chaincode definition on peer0.org2 on channel 'mychannel'... =====================
+ peer lifecycle chaincode checkcommitreadiness --channelID mychannel --name mycc --version 1 --sequence 1 --output json --init-required
Attempting to check the commit readiness of the chaincode definition on peer0.org2 ...3 secs
+ res=0
+ set +x

{
	"approvals": {
		"Org1MSP": true,
		"Org2MSP": true
	}
}
===================== Checking the commit readiness of the chaincode definition successful on peer0.org2 on channel 'mychannel' =====================
+ peer lifecycle chaincode commit -o orderer.example.com:7050 --tls true --cafile /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem --channelID mychannel --name mycc --peerAddresses peer0.org1.example.com:7051 --tlsRootCertFiles /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt --peerAddresses peer0.org2.example.com:9051 --tlsRootCertFiles /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt --version 1 --sequence 1 --init-required
+ res=0
+ set +x
2020-03-18 08:33:47.436 UTC [chaincodeCmd] ClientWait -> INFO 001 txid [501f9798ad6ae0eaf566114920557c5ac0e793af13f92a8020e65821492ef7ec] committed with status (VALID) at peer0.org2.example.com:9051
2020-03-18 08:33:47.452 UTC [chaincodeCmd] ClientWait -> INFO 002 txid [501f9798ad6ae0eaf566114920557c5ac0e793af13f92a8020e65821492ef7ec] committed with status (VALID) at peer0.org1.example.com:7051
===================== Chaincode definition committed on channel 'mychannel' =====================

===================== Querying chaincode definition on peer0.org1 on channel 'mychannel'... =====================
Attempting to Query committed status on peer0.org1 ...3 secs
+ peer lifecycle chaincode querycommitted --channelID mychannel --name mycc
+ res=0
+ set +x

Committed chaincode definition for chaincode 'mycc' on channel 'mychannel':
Version: 1, Sequence: 1, Endorsement Plugin: escc, Validation Plugin: vscc, Approvals: [Org1MSP: true, Org2MSP: true]
===================== Query chaincode definition successful on peer0.org1 on channel 'mychannel' =====================
===================== Querying chaincode definition on peer0.org2 on channel 'mychannel'... =====================
+ peer lifecycle chaincode querycommitted --channelID mychannel --name mycc
Attempting to Query committed status on peer0.org2 ...3 secs
+ res=0
+ set +x

Committed chaincode definition for chaincode 'mycc' on channel 'mychannel':
Version: 1, Sequence: 1, Endorsement Plugin: escc, Validation Plugin: vscc, Approvals: [Org1MSP: true, Org2MSP: true]
===================== Query chaincode definition successful on peer0.org2 on channel 'mychannel' =====================
+ peer chaincode invoke -o orderer.example.com:7050 --tls true --cafile /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem -C mychannel -n mycc --peerAddresses peer0.org1.example.com:7051 --tlsRootCertFiles /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt --peerAddresses peer0.org2.example.com:9051 --tlsRootCertFiles /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt --isInit -c '{"Args":["Init","a","100","b","100"]}'
+ res=0
+ set +x
2020-03-18 08:33:54.365 UTC [chaincodeCmd] chaincodeInvokeOrQuery -> INFO 001 Chaincode invoke successful. result: status:200
===================== Invoke transaction successful on peer0.org1 peer0.org2 on channel 'mychannel' =====================

Querying chaincode on peer0.org1...
===================== Querying on peer0.org1 on channel 'mychannel'... =====================
+ peer chaincode query -C mychannel -n mycc -c '{"Args":["query","a"]}'
Attempting to Query peer0.org1 ...3 secs
+ res=0
+ set +x

100
===================== Query successful on peer0.org1 on channel 'mychannel' =====================
Sending invoke transaction on peer0.org1 peer0.org2...
+ peer chaincode invoke -o orderer.example.com:7050 --tls true --cafile /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem -C mychannel -n mycc --peerAddresses peer0.org1.example.com:7051 --tlsRootCertFiles /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt --peerAddresses peer0.org2.example.com:9051 --tlsRootCertFiles /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt -c '{"Args":["invoke","a","b","10"]}'
+ res=0
+ set +x
2020-03-18 08:33:57.957 UTC [chaincodeCmd] chaincodeInvokeOrQuery -> INFO 001 Chaincode invoke successful. result: status:200
===================== Invoke transaction successful on peer0.org1 peer0.org2 on channel 'mychannel' =====================

Querying chaincode on peer0.org1...
===================== Querying on peer0.org1 on channel 'mychannel'... =====================
Attempting to Query peer0.org1 ...3 secs
+ peer chaincode query -C mychannel -n mycc -c '{"Args":["query","a"]}'
+ res=0
+ set +x

90
===================== Query successful on peer0.org1 on channel 'mychannel' =====================
Installing chaincode on peer1.org2...
+ peer lifecycle chaincode install mycc.tar.gz
+ res=0
+ set +x
2020-03-18 08:34:19.078 UTC [cli.lifecycle.chaincode] submitInstallProposal -> INFO 001 Installed remotely: response:<status:200 payload:"\nGmycc_1:2d8de5706ec819943d4a98931bbb377a66b669092ca559a61d0e6721e38af7ca\022\006mycc_1" >
2020-03-18 08:34:19.078 UTC [cli.lifecycle.chaincode] submitInstallProposal -> INFO 002 Chaincode code package identifier: mycc_1:2d8de5706ec819943d4a98931bbb377a66b669092ca559a61d0e6721e38af7ca
===================== Chaincode is installed on peer1.org2 =====================

Querying chaincode on peer1.org2...
===================== Querying on peer1.org2 on channel 'mychannel'... =====================
Attempting to Query peer1.org2 ...3 secs
+ peer chaincode query -C mychannel -n mycc -c '{"Args":["query","a"]}'
+ res=0
+ set +x

90
===================== Query successful on peer1.org2 on channel 'mychannel' =====================

========= All GOOD, BYFN execution completed ===========


 _____   _   _   ____
| ____| | \ | | |  _ \
|  _|   |  \| | | | | |
| |___  | |\  | | |_| |
|_____| |_| \_| |____/

基于TCP的网络优化器 Q&A

A. 使用篇

配置文件在哪里?

在这里:https://mhy12345.xyz/technology/tcp-optimizer-configures/

不同的平台用什么客户端?

TROJAN

  • 第一步:客户端下载
  • 其他平台客户端下载链接合集:https://tlanyan.me/trojan-clients-download/
  • 第二步:修改配置文件
    • 详细配置文件见文末<Trojan客户端配置文件>,主要是下面几个有变化:
    • 修改完配置文件记得运行trojan客户端,使其监听本地1088端口
  • 第三步:安装浏览器插件ProxyOmega
    • 安装过程中可能遇到INVALID HEADER的问题:怎么解决
    • 全局模式:
      1. 右键浏览器ProxyOmega插件的图标(那个圆圈⭕️),选择Options
      2. 新建 Proxy Profile,取名为Trojan
      3. 依次填写Socks5, 127.0.0.1,1088
      4. Apply Change
      5. 右键浏览器插件栏的那个圈⭕️,然后选择Trojan
    • 部分代理模式(Pac模式)
      1. 右键浏览器ProxyOmega插件的图标(那个圆圈⭕️),选择Options
      2. 新建PAC Profile,取名Trojan-autoswitch
      3. 在PacScripts那个框里面填入附录<Trojan-浏览器插件Pac配置文件>内容
      4. Apply Change
      5. 右键浏览器插件栏的那个圈⭕️,然后选择Trojan-autoswitch

shadowsocks

v2ray

谷歌学术被显示无法访问?

谷歌学术镜像:https://google.mhy12345.xyz/scholar
谷歌搜索镜像:https://google.mhy12345.xyz/
密码:见密码公示页面

其他谷歌学术镜像:https://ac.scmor.com/

如果出现下载缓慢的情况怎么办?

对于点击即可下载的链接,如果下载缓慢,可以尝试右键“拷贝链接地址”,然后将拷贝到的链接放到迅雷下载里面。

终端需要下载资源,但是被墙了怎么办?

表现为类似于curl: (7) Failed to connect to raw.githubusercontent.com port 443: Operation timed out 这样的报错信息,意味着终端访问的资源被墙了。

  1. 打开翻墙软件
  2. 右键翻墙软件图标,点击Copy HTTP Proxy Shell Export Line,此时会将一个终端命令拷贝到剪切板中
  3. 打开终端,将这句话输入,再执行你想要的命令。
# 剪贴板中的命令可能是这样
export http_proxy=http://127.0.0.1:1087;export https_proxy=http://127.0.0.1:1087;

注意大量使用代理下载可能对服务器造成巨大负担,尽量避免这种情况。

如果你水平比较高,想自己DIY

使用ssh root@us.digitalocean.server.mhy12345.xyz命令登录服务器,开一个自己的用户,在上面自己玩。注意别把root用户的进程kill了就行。

B. 搭建篇

原理

防火墙(great firewall, gfw)的原理是过滤你向外网黑名单服务器(如google, youtube服务器)发送的网络请求包,使得本地无法直接连到这些服务器。

我们需要在大陆地区以外,买一个服务器作为中转。这些售卖云服务器的商家就是服务商(如google cloud,vultr等)

在服务器上面安装指定的服务端软件(如v2ray-server, shadowsocks-server),并在自己的电脑上装对应的客户端(如v2ray-client, shadowsocks-client),那么本地需要翻墙时,就会将本地的流量经由客户端加密发送至服务端,由服务端去连接我们需要的服务。这样由于出国的流量是完全伪装加密的,所以不会被防火墙拦截。

有哪些服务商可以选择?

  • google cloud:有1000刀的免费额度,需要翻墙,网络较顺畅
  • vultr:网站本身不需要翻墙
  • digitalocean:不需要翻墙,但链接不稳定
  • amazon aws:有一年的免费额度

有哪些翻墙软件可以选择?

  • 最原始的vpn软件,不推荐
  • shadowsocks,一个专门用于翻墙的软件,不过由于使用范围太广,gfw已经存在一些识别shadowsocks的算法,可以识别ss的流量,并且封闭对应的服务器或端口。
  • shadowsocks-libev,一个重写shadowsocks的软件,在ss的基础上,添加了aes-256-gcm等加密方式,更难被识别
  • v2ray,近一两年新出现的翻墙软件,拥有大量的伪装算法,比较靠谱
  • trojan,需要有自己的域名,搭起来相对技术要求高一些,搭建教程:https://bandwh.com/kxsw/50.html ,我自己使用的脚本:https://github.com/mhy12345/Configures/blob/master/vps/trojan_install.sh

搭建v2ray方式是什么?

使用如下脚本——

cd ~
wget https://git.io/v2ray.sh
. v2ray.sh

随意选择看起来靠谱的加密方式以及端口。最后使用

v2ray url

获得v2ray的连接。

在这个过程中有一些注意事项,包括

  • 需要在服务器提供商的页面上面开放代理端口的IN/OUT
  • 需要在服务器内部的防火墙(firewall/ufw/iptables)上面开放代理端口的IN/OUT
  • 需要使用crontab设置定时重启v2ray restart

常用的查错思路

翻墙服务器搭建本身很简单,主要是需要和gfw斗智斗勇,如果你发现翻不出去,可以思考下面的问题——

  1. 这个服务器曾经成功翻墙过吗?
    1. 如果从来没有成功过
      1. 端口被服务器提供商禁止了
      2. 系统防火墙禁止了
    2. 如果曾经有一段时间是好的
      1. 服务器的端口被gfw断了,可以考虑换一个端口
      2. 服务器内部的转发进程卡死了,可以考虑重启进程
      3. 服务器网络状态不佳,可能表现在ping不通或者丢包率高,考虑
        1. 等一段时间看能不能恢复
        2. 升级为更高的配置
        3. 重新申请一个新的实例,看新的ip是不是会好一些
        4. 如果这个服务器提供商所有ip都很糟糕,考虑换一家服务商了
  2. 这个服务器现在连接通畅吗?
    1. 可以在本机 ping 服务器地址或域名
      1. 如果ping不通
        1. 使用的是域名的方式,则可能是dns服务器有问题,尝试换一个dns服务器
        2. 可能是服务器运营商的防火墙阻止了icmp包
      2. 如果ping通了,但是存在较高的丢包率,则可能是你现在选的服务器配置太低,考虑升级服务器
      3. 如果ping通了,但是延迟超过300ms,那么也有可能是选的服务器配置太低
      4. 如果ping通了,延迟也较小,那么基本上还是1.1的情况
    2. 使用服务器telnet翻墙服务的端口
      1. 成功(看到Escape character is '^]'.这句话):翻墙服务成功启动
      2. 失败:翻墙服务设置有问题
    3. 使用本机telnet翻墙服务的端口
      1. 成功:如果同时ping很通畅的话,说明是远程配置正确,本地客户端设置有问题
      2. 失败:如果不能ping通,参见2.1.1,否则是
        1. 端口被服务器提供商禁用了
        2. 端口gfw墙了,考虑换端口

非常时刻,全部外网服务器都被封了怎么办?

如果你有国内服务器A,用于翻墙但是端口被封的服务器B,可以使用ssh隧道的方式连接服务器A,B

在服务器A里面的home目录放置下面的脚本——

#!/bin/bash
while true
do
    date
    sleep 1m
done

在服务器B里面开一个后台进程,运行下面的脚本——

#!/bin/bash
while true
do
    ssh -R 33555:localhost:8389 <USER@HOSTNAME OF SERVER A> -p 2233 bash idle.sh & ssh_pid=!     echo ssh pid isssh_pid
    while true
    do
        sleep 30s
        curl http://mhy12345.xyz:33555
        if [ ? != 52 ]         then             echo Connection broken             kill -9ssh_pid
            kill -9 ssh_pid             kill -9ssh_pid
            kill -9 $ssh_pid
            break
        else
            date
        fi
    done
done

在这样的设置后,我们本地就不需要连接外网服务器B的8389端口,而是直接连接内网服务器的端口8389实现翻墙。这样的做法在中间凭空多了一个跳转,大幅度提高了延迟。所以网速会非常慢。

附件

Trojan客户端配置文件

{
     "run_type": "client",
     "local_addr": "127.0.0.1",
     "local_port": 1088,
     "remote_addr": "ipv4.vpn3.mhy12345.xyz",
     "remote_port": 443,
     "password": [
         "<见群里发的>"
     ],
     "log_level": 1,
     "ssl": {
         "verify": true,
         "verify_hostname": true,
         "cert": "",
         "cipher": "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA:AES128-SHA:AES256-SHA:DES-CBC3-SHA",
         "cipher_tls13": "TLS_AES_128_GCM_SHA256:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_256_GCM_SHA384",
         "sni": "",
         "alpn": [
             "h2",
             "http/1.1"
         ],
         "reuse_session": true,
         "session_ticket": false,
         "curves": ""
     },
     "tcp": {
         "no_delay": true,
         "keep_alive": true,
         "reuse_port": false,
         "fast_open": false,
         "fast_open_qlen": 20
     }
 }

Trojan-浏览器插件Pac配置文件

下载文件,用记事本打开,然后将内容复制出来。

球坐标系角距离(angular distance)下的KD树最近点查询算法

代码地址:https://github.com/mhy12345/angular-kdtree

给定一个坐标集合S,对于若干询问坐标p,分别求出该坐标在坐标集合中最邻近(K临近)的坐标。

KNN算法

问题概述

在K临近算法是一个经典的问题,通常在平面欧几里得坐标系下,我们使用KDTree这个算法来解决,时间复杂度为单次查询O(\sqrt{N})。现在,我们需要将这个KD树算法扩展到球坐标系下,来解决球坐标系中的最近点查询。具体来讲,定义问题为——

通过经度纬度定义球坐标系坐标点,并给定角距离公式 angular_separation 如下,这里的距离即天文学中的天体视角

double angular_separation(double lon1, double lat1, double lon2, double lat2) {
    double sdlon = sin(lon2 - lon1);
    double cdlon = cos(lon2 - lon1);
    double slat1 = sin(lat1);
    double slat2 = sin(lat2);
    double clat1 = cos(lat1);
    double clat2 = cos(lat2);

    double num1 = clat2 * sdlon;
    double num2 = clat1 * slat2 - slat1 * clat2 * cdlon;
    double denominator = slat1 * slat2 + clat1 * clat2 * cdlon;
    return atan2(hypot(num1, num2), denominator);
}

我们对于每个询问坐标q,查询到与q距离最近的一个集合S中的坐标。

实现思路

与普通KD树相似,使用横纵轴轮流分割的方式建树。即,第一层将所有点按照维度大小对半分,第二层在第一层的基础上按照经度对半分。这样,KD树每一个节点都维护一个球坐标bounding-box内的坐标集合。对于每一次询问,只需要递归对每个子树遍历查询,并通过计算坐标与bounding-box的最短角距离来进行遍历剪枝。

使用样例

C++语言

#include "kdtree.h"
#include <iostream>
using namespace std;

int main() {
    KDTree kdt = KDTree(); // 新建一个KDTree实例
    Point point_to_insert = Point(
        random()%10000/10000.0*PI*2,  // x, 维度坐标,范围(0, 2pi)
        random()%10000/10000.0*PI - PI/2,  // y, 精度坐标,范围(-pi/2, pi/2)
        NULL // tag, 一个类型为const char*的标签,可用于储存必要的信息
        );
    kdt.Insert(point_to_insert);
    Point point_to_query = Point(0,0,NULL);
    pair<double,const Point*> kdt_res = kdt.Search(
        point_to_query,  // pt, 待查询的坐标, tag可为空,
        0.1 // r, 搜索距离,即查询角距离小于该值的区域
        );
    if (kdt_res.second) { // NULL表示没有找到,反之储存结果
        cout<<"Distance : "<<kdt_res.first<<endl;
        cout<<Point : "<<kdt_res.second->x<<" "<<kdt_res.second.y<<endl;
    }
}

Python语言

import ctypes
from ctypes import *

so = cdll.LoadLibrary   
lib = so("/path/to/kdt_toolbox.so")
lib.KDTBatchInsert.argtypes = [c_int, POINTER(c_double), POINTER(c_double), POINTER(c_char_p)]
lib.KDTInsert.argtypes = [c_double, c_double, c_char_p]
lib.KDTSearch.argtypes = [c_double,c_double]
lib.KDTSearch.restype = c_char_p

xs = np.array(... ,dtype=np.double)
ys = np.array(... ,dtype=np.double)
sid = list(map(lambda x:x.encode('ascii'), ...))

char_p_arr = ctypes.c_char_p * len(data)
sid_p = char_p_arr(*sid)

xs_p = xs.ctypes.data_as(POINTER(c_double))
ys_p = ys.ctypes.data_as(POINTER(c_double))

lib.KDTInit()
args = c_int(len(data)),xs_p, ys_p, sid_p
lib.KDTBatchInsert(*args)

args = c_double(xs[0]),c_double(ys[0])
res = lib.KDTSearch(*args)
assert(res.decode('ascii') == str(sid[0]))

如果你觉得有用,记得点star哦~