Machine Learning – mhy12345

论文链接：Deep Learning Stock Volatility with Google Domestic Trends

课件链接：mhy-Deep Learning Stock Volatility with Google Domestic Trends_pptx

概述

这篇文章核心目标是，通过长短期记忆循环神经网络（LSTM）预测股市波动率，其中输入数据依赖于Google Domestic Trend这样一个搜索量指数。

Google Domestic Trend里面提取了Google中每一个关键词相对于时间的搜索量变化，当然，如果说联想到国内数据，百度指数提供了相似的操作

首先，选取若干具有代表性的词语，如bankruptcy,auto trading, travel等，将其热度指数同股价一起输入LSTM，来预测波动率。

波动率的计算公式比较有意思，

$\sigma = 0.511(u-d)^2 - 0.019[c(u+d)-2ud]-0.383c^2$

其中 $u=log(\frac{High}{Open})$ ， $d=log(\frac{Low}{Open})$ ， $c=log(\frac{Close}{Open})$ .

即该公式仅仅依赖于四个价格，他和MACD用不同的算法导出了几乎相似的指数，这也是一个非常美妙的地方。

本文在预测 $\sigma$ 之外，也同时尝试了预测价格变化量，即

$r_i = log(\frac{Close_i}{Close_{i-1}})$

只是一个label不同的问题，就不详述了。

技巧1：平滑

细粒度的数据存在较大的杂音，故希望通过增大粒度来去除杂音。这里增大粒度就存在值得合并问题

令 $\Delta t$ 为平滑的时间区间.

收益率由于已经取过log了，所以可以直接求和

$r_i^{\Delta t} = \sum^{i \Delta t}_{t=(i-1)\Delta t +1} r_t$

搜索量合并是一个算数平均的过程

$d_i^{\Delta t} = \frac{1}{\Delta t} \sum^{i \Delta t}_{t=(i-1)\Delta t +1} d_t$

波动率是一个几何平均过程

$\sigma_i^{\Delta t} = \sqrt{\sum^{i \Delta t}_{t=(i-1)\Delta t +1} \sigma_t^2}$

技巧2：归一

对于任意序列A，都有归一化公式

$Z = \frac{A_i - mean(A)}{std(A)}$

而对于时序A，可以加一个滑动窗口时长K进行仿照

$Z^A_{k,i} = \frac{A_i - mean(A_{i-k:i})}{std(A_{i-k:i})}$

这样的归一方法，既保证了短期趋势的完整复制，还可以避免长期趋势导致的值不平均。

结论

如其他论文一样，这里还是借助其他模型进行比较，结论是：比其他模型效果更好。不过恐怕也是五十步笑百步了吧~

就这个问题而言，如果预测 $r_i$ ，可能会比较有实用价值，但是预测效果会很差，毕竟确实想不出来涨跌会和这东西有什么关系。但是如果是预测 $\sigma_i$ ，虽然效果不错，但是实用价值又不太高。总之感觉很鸡肋啊。

RNN用于处理不定长的序列数据，而简谐波预测就是一个很好的例子，显然，对于机器学习，sin(x)、cos(x)、a*sin(x) + b*cos(x)甚至x*sin(x)都没有任何难度上的差别。

那么我们就用RNN来预测吧！

（CNMB……由于一个偶然的错误，我发现这个问题直接线性规划类型的神经网络就可以很精确的计算出来！！！而且LSTM根本比不上！！！）

考虑

sin(a(k-1)+b)、sin(ak+b) 和sin(a(k+1)+b)的关系

sin(a(k-1)+b) = sin(ak+b-a) = sin(ak+b)cos(a) – cos(ak+b)sin(a)

sin(a(k+1)+b) = sin(ak+b+a) = sin(ak+b)cos(a) + cos(ak+b)sin(a)

得

sin(a(k+1)+b) = -sin(a(k-1)+b) + 2sin(ak+b)cos(a)

其中由于a我取的是整数，所以非常容易估计，于是实际上sin数列的每一项可以由前两项线性表示出来……

即使a不是整数，也可通过三角带换，得到近似于线性表达的公式

但是现在就出现这样一个问题。

既然是如此简单的问题，LSTM的表现为啥这么渣呢？？？

~~原因可能如下：~~

~~LSTM变量太多了，训练会出现梯度消失的问题~~
~~使用LSTM，倒数第二项作用于倒数第一项，中间有若干非线性层，导致本来非常简单的线性组合变得非常难以模拟了！~~
写错了……

脑残的把最后一层加了一个sigmoid函数，然而并不知道sigmoid的返回值是一个正数，并不包含sin(x)的值域区间

import tensorflow as tf
import numpy as np
import talib
import random
from matplotlib import pyplot

max_length = 20
total_branches = 10010
batch_size = 31
hidden_size = 16
start_position = 10
test_batch_index = 0
num_input_classes = 1
difficulity = 10
total_layers = 1

test_x = None
test_y = None
test_seqlen = None

def gen_sine_wave(length):
    L = random.randint(1,difficulity)
    R = random.randint(1,difficulity)
    if L>R:
        L,R = R,L
    x = np.linspace(L,R,length+1)
    y = np.sin(x)
    return y[:-1],y[-1]

def generate_batches():
    global test_x,test_y,test_seqlen
    test_x = []
    test_y = []
    test_seqlen = []
    test_batch_len = 0
    for i in range(0,total_branches):
        length = random.randint(start_position,max_length)
        x,y = gen_sine_wave(length)
        x = np.append(np.array([t for t in x]),np.array([0 for i in range(max_length-length)]))
        x = np.reshape(x,[max_length,num_input_classes])
        test_seqlen.append(length)
        test_x.append(x)
        test_y.append(y)

def next_batch(total_branches = total_branches):
    global test_batch_index,test_x,test_y,test_seqlen
    if test_batch_index == total_branches:
        test_batch_index = 0
    xs = test_x[test_batch_index:min(test_batch_index+batch_size,total_branches)]
    ys = test_y[test_batch_index:min(test_batch_index+batch_size,total_branches)]
    seqlens = test_seqlen[test_batch_index:min(test_batch_index+batch_size,total_branches)]
    test_batch_index = min(test_batch_index + batch_size,total_branches)
    return xs,ys,seqlens


if __name__ == '__main__':
    generate_batches()
    #sess = tf.InteractiveSession()
    input_data = tf.placeholder(tf.float32,[None,max_length,num_input_classes])
    input_seqlen = tf.placeholder(tf.int32,[None])
    input_labels = tf.placeholder(tf.float32,[None])
    keep_prob = tf.placeholder(tf.float32)
    current_batch_size = tf.shape(input_data)[0]

    with tf.name_scope('lstm_layer'):
        method = 'lstm'
        data = input_data
        print(data.shape)#10*20*1
        if method == 'lstm':
            def create_lstm():
                lstm_cell = tf.contrib.rnn.BasicLSTMCell(hidden_size,state_is_tuple = True)
                #lstm_cell = tf.nn.rnn_cell.DropoutWrapper(lstm_cell, output_keep_prob=keep_prob,state_keep_prob=keep_prob)
                return lstm_cell
            lstm_cell = create_lstm()
            #lstm_cell = tf.nn.rnn_cell.MultiRNNCell([create_lstm() for i in range(total_layers)], state_is_tuple=True)
            #state = lstm_cell.zero_state(current_batch_size,tf.float32)
            data = tf.unstack(data,axis=1)
            outputs,state = tf.nn.static_rnn(lstm_cell,data,dtype=tf.float32,sequence_length=input_seqlen)
            outputs = tf.stack(outputs,axis=1)
            #outputs =  tf.transpose(outputs,[1,0,2])
        else:
            gru_cell = tf.contrib.rnn.GRUCell(hidden_size)
            outputs,state = tf.nn.dynamic_rnn(gru_cell,data,dtype=tf.float32,sequence_length=input_seqlen)

        print(outputs.shape)
        output = tf.gather_nd(outputs, tf.stack([tf.range(current_batch_size), input_seqlen-1], axis=1))

    with tf.name_scope('final_layer'):
        final_w = tf.Variable(tf.truncated_normal([hidden_size,1],.1))
        tf.summary.histogram('weight',final_w)
        final_b = tf.Variable(tf.constant(.1,shape=[1]))
        tf.summary.histogram('bias',final_b)

    with tf.name_scope('output_layer'):
        result = tf.tanh(tf.matmul(output,final_w) + final_b)
        print(result.shape)
        result = tf.reshape(result,[-1])
        loss = tf.reduce_mean(tf.losses.mean_squared_error(labels = input_labels,predictions=result))
        tf.summary.scalar('loss',loss)

    optimizer = tf.train.AdamOptimizer().minimize(loss)

    with tf.Session() as sess:
        sess.run(tf.global_variables_initializer())
        merged = tf.summary.merge_all()
        writer = tf.summary.FileWriter("logs/",sess.graph)

        steps = 0
        while True:
            batch_x , batch_y, batch_seqlen = next_batch()
            sess.run(optimizer, feed_dict={input_data:batch_x,
                input_labels:batch_y,
                input_seqlen:batch_seqlen,
                keep_prob:.95})
            if steps%50 == 0:
                print(steps,sess.run(loss,feed_dict={input_data:batch_x,input_labels:batch_y,input_seqlen:batch_seqlen,keep_prob:1}))
                print("PREDICT",sess.run(result,feed_dict={input_data:batch_x,input_labels:batch_y,input_seqlen:batch_seqlen,keep_prob:1}))
                print("LABEL",sess.run(input_labels,feed_dict={input_data:batch_x,input_labels:batch_y,input_seqlen:batch_seqlen,keep_prob:1}))
                merged_result = sess.run(merged, feed_dict={input_data:batch_x,
                    input_labels: batch_y,
                    input_seqlen: batch_seqlen,
                    keep_prob:1})
                writer.add_summary(merged_result,steps)
            steps += 1
            if steps%1000 == 0:
                batch_x , batch_y, batch_seqlen = next_batch()
                current_result = sess.run(result,feed_dict={input_data:batch_x,input_seqlen:batch_seqlen,keep_prob:1})
                X = batch_x[0][:batch_seqlen[0]]
                Y = current_result[0]
                Z = batch_y[0]
                plt = pyplot.figure()
                pyplot.plot(np.linspace(1,len(X),len(X)),X)
                pyplot.plot(len(X)+1,Y,'r+')
                pyplot.plot(len(X)+1,Z,'r^')
                pyplot.show()

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

import tensorflow as tf

import numpy as np

import talib

import random

from matplotlib import pyplot

max_length = 20

total_branches = 10010

batch_size = 31

hidden_size = 16

start_position = 10

test_batch_index = 0

num_input_classes = 1

difficulity = 10

total_layers = 1

test_x = None

test_y = None

test_seqlen = None

def gen_sine_wave(length):

L = random.randint(1,difficulity)

R = random.randint(1,difficulity)

if L>R:

L,R = R,L

x = np.linspace(L,R,length+1)

y = np.sin(x)

return y[:-1],y[-1]

def generate_batches():

global test_x,test_y,test_seqlen

test_x = []

test_y = []

test_seqlen = []

test_batch_len = 0

for i in range(0,total_branches):

length = random.randint(start_position,max_length)

x,y = gen_sine_wave(length)

x = np.append(np.array([t for t in x]),np.array([0 for i in range(max_length-length)]))

x = np.reshape(x,[max_length,num_input_classes])

test_seqlen.append(length)

test_x.append(x)

test_y.append(y)

def next_batch(total_branches = total_branches):

global test_batch_index,test_x,test_y,test_seqlen

if test_batch_index == total_branches:

test_batch_index = 0

xs = test_x[test_batch_index:min(test_batch_index+batch_size,total_branches)]

ys = test_y[test_batch_index:min(test_batch_index+batch_size,total_branches)]

seqlens = test_seqlen[test_batch_index:min(test_batch_index+batch_size,total_branches)]

test_batch_index = min(test_batch_index + batch_size,total_branches)

return xs,ys,seqlens

if __name__ == '__main__':

generate_batches()

#sess = tf.InteractiveSession()

input_data = tf.placeholder(tf.float32,[None,max_length,num_input_classes])

input_seqlen = tf.placeholder(tf.int32,[None])

input_labels = tf.placeholder(tf.float32,[None])

keep_prob = tf.placeholder(tf.float32)

current_batch_size = tf.shape(input_data)[0]

with tf.name_scope('lstm_layer'):

method = 'lstm'

data = input_data

print(data.shape)#10*20*1

if method == 'lstm':

def create_lstm():

lstm_cell = tf.contrib.rnn.BasicLSTMCell(hidden_size,state_is_tuple = True)

#lstm_cell = tf.nn.rnn_cell.DropoutWrapper(lstm_cell, output_keep_prob=keep_prob,state_keep_prob=keep_prob)

return lstm_cell

lstm_cell = create_lstm()

#lstm_cell = tf.nn.rnn_cell.MultiRNNCell([create_lstm() for i in range(total_layers)], state_is_tuple=True)

#state = lstm_cell.zero_state(current_batch_size,tf.float32)

data = tf.unstack(data,axis=1)

outputs,state = tf.nn.static_rnn(lstm_cell,data,dtype=tf.float32,sequence_length=input_seqlen)

outputs = tf.stack(outputs,axis=1)

#outputs = tf.transpose(outputs,[1,0,2])

else:

gru_cell = tf.contrib.rnn.GRUCell(hidden_size)

outputs,state = tf.nn.dynamic_rnn(gru_cell,data,dtype=tf.float32,sequence_length=input_seqlen)

print(outputs.shape)

output = tf.gather_nd(outputs, tf.stack([tf.range(current_batch_size), input_seqlen-1], axis=1))

with tf.name_scope('final_layer'):

final_w = tf.Variable(tf.truncated_normal([hidden_size,1],.1))

tf.summary.histogram('weight',final_w)

final_b = tf.Variable(tf.constant(.1,shape=[1]))

tf.summary.histogram('bias',final_b)

with tf.name_scope('output_layer'):

result = tf.tanh(tf.matmul(output,final_w) + final_b)

print(result.shape)

result = tf.reshape(result,[-1])

loss = tf.reduce_mean(tf.losses.mean_squared_error(labels = input_labels,predictions=result))

tf.summary.scalar('loss',loss)

optimizer = tf.train.AdamOptimizer().minimize(loss)

with tf.Session() as sess:

sess.run(tf.global_variables_initializer())

merged = tf.summary.merge_all()

writer = tf.summary.FileWriter("logs/",sess.graph)

steps = 0

while True:

batch_x , batch_y, batch_seqlen = next_batch()

sess.run(optimizer, feed_dict={input_data:batch_x,

input_labels:batch_y,

input_seqlen:batch_seqlen,

keep_prob:.95})

if steps%50 == 0:

print(steps,sess.run(loss,feed_dict={input_data:batch_x,input_labels:batch_y,input_seqlen:batch_seqlen,keep_prob:1}))

print("PREDICT",sess.run(result,feed_dict={input_data:batch_x,input_labels:batch_y,input_seqlen:batch_seqlen,keep_prob:1}))

print("LABEL",sess.run(input_labels,feed_dict={input_data:batch_x,input_labels:batch_y,input_seqlen:batch_seqlen,keep_prob:1}))

merged_result = sess.run(merged, feed_dict={input_data:batch_x,

input_labels: batch_y,

input_seqlen: batch_seqlen,

keep_prob:1})

writer.add_summary(merged_result,steps)

steps += 1

if steps%1000 == 0:

batch_x , batch_y, batch_seqlen = next_batch()

current_result = sess.run(result,feed_dict={input_data:batch_x,input_seqlen:batch_seqlen,keep_prob:1})

X = batch_x[0][:batch_seqlen[0]]

Y = current_result[0]

Z = batch_y[0]

plt = pyplot.figure()

pyplot.plot(np.linspace(1,len(X),len(X)),X)

pyplot.plot(len(X)+1,Y,'r+')

pyplot.plot(len(X)+1,Z,'r^')

pyplot.show()

标签：Machine Learning

Alpha-Nebula：Deep Learning Stock Volatility with Google Domestic Trends

概述

技巧1：平滑

技巧2：归一

结论

Alpha-Nebula：Short-term stock price prediction based on echo state networks

Echo State Network

使用ESN预测短期股价

想法与疑问

RNN学习笔记之回声状态网络

RNN学习笔记之利用tensorflow-lstm实现sin预测