利用矩阵运算仿神经网络

1.利用矩阵运算仿神经网络的信息传导

图后补

y1 = activation_functionx1 * w11 + x2 * w21 + x3 * w31 +b1
y2 = activation_functionx1 * w12 + x2 * w22 + x3 * w32 +b2

以上公式可以整合成一个矩阵公式

[y1 y2] = activation( [x1 x2 x3] * W + [b1 b2] )

W是一个2*3的矩阵

另外一种形式的矩阵公式

y = activationX * W + b 

中文解析

输出 = 激活函数(输入 * 权重 + 偏差)

2.以TensorFlow张量运仿神经网络

import tensorflow as tf
# 使用TensorFlow张量运算模拟以上公式
X = tf.Variable([[0.4,0.2,0.4]])
W = tf.Variable([
    [-0.5,-0.2],
    [-0.3,0.4],
    [-0.5,0.2]
])
b = tf.Variable([[0.1,0.2]])
XWb = tf.matmul(X,W) + b
y = tf.nn.relu(tf.matmul(X,W)+b)

with tf.Session() as sess:
    init = tf.global_variables_initializer()
    sess.run(init)
    print('XWb : ')
    print(sess.run(XWb))
    print('y : ')
    print(sess.run(y))
XWb : 
[[-0.36000004  0.28      ]]
y : 
[[-0\.    0.28]]

在TensorFlow使用tf.nn.relu引用ReLU激活函数。ReLU的特色是,如果小于0转换后是0,如果大于0输出等于输入。所以第一个元素-0.36经过ReLU转后后是0,第2个元素0.28经过ReLU转换后仍然是0.28。

接下来我们使用sigmoid激活函数

X = tf.Variable([[0.4,0.2,0.4]])
W = tf.Variable([
    [-0.5,-0.2],
    [-0.3,0.4],
    [-0.5,0.2]
])
b = tf.Variable([[0.1,0.2]])
XWb = tf.matmul(X,W) + b
y = tf.nn.sigmoid(tf.matmul(X,W)+b)

with tf.Session() as sess:
    init = tf.global_variables_initializer()
    sess.run(init)
    print('XWb : ')
    print(sess.run(XWb))
    print('y : ')
    print(sess.run(y))
XWb : 
[[-0.36000004  0.28      ]]
y : 
[[0.41095954 0.5695462 ]]

接下来我们使用正态分布随机数生存权重与偏差的初始值

W = tf.Variable(tf.random_normal([3,2]))
b = tf.Variable(tf.random_normal([1,2]))
X = tf.Variable([[0.4,0.2,0.4]])
y = tf.nn.relu(tf.matmul(X,W)+b)

with tf.Session() as sess:
    init = tf.global_variables_initializer()
    sess.run(init)
    print('b : \n',sess.run(b))
    print('W : \n',sess.run(W))
    print('y : \n',sess.run(y))
b : 
 [[0.43149146 0.20747285]]
W : 
 [[ 0.6440627   0.04269069]
 [ 1.8402126   0.173751  ]
 [-0.6833311  -0.45362884]]
y : 
 [[0.7838266  0.07784778]]
# 换一种写法
W = tf.Variable(tf.random_normal([3,2]))
b = tf.Variable(tf.random_normal([1,2]))
X = tf.Variable([[0.4,0.2,0.4]])
y = tf.nn.relu(tf.matmul(X,W)+b)

with tf.Session() as sess:
    init = tf.global_variables_initializer()
    sess.run(init)
    (_b,_W,_y) = sess.run((b,W,y))
    print('b:')
    print(_b)
    print('W:')
    print(_W)
    print('y:')
    print(_y)
b:
[[0.5062327  0.05478674]]
W:
[[-0.2955449   0.9855861 ]
 [ 0.08898462  0.48763534]
 [ 1.0256839  -1.2430311 ]]
y:
[[0.8160852  0.04933578]]

大家会好奇,什么是正太随机数,我们写个简单的程序来说明

ts_norm = tf.random_normal([1000])
with tf.Session() as session:
    norm_data = ts_norm.eval()
print(norm_data[:5])
[ 0.33918783  0.30642796  1.339344   -0.01829625 -0.43177664]
# 画出图来
import numpy as np
import matplotlib.pyplot as plt
plt.hist(norm_data)
plt.show()

output_13_0.png

3.使用placeholder传入x值

将之前的X(x1,x2,x3)神经网络的输入更改成使用placeholder的方式传入进行运算。我们传入1 * 3 的二维数组

# placeholder传入二维数组X = [[0.4,0.2,0.4]]

W = tf.Variable(tf.random_normal([3, 2]))
b = tf.Variable(tf.random_normal([1, 2]))
X = tf.placeholder("float", [None,3])
y=tf.nn.relu(tf.matmul(X,W)+b)
with tf.Session() as sess:
    init = tf.global_variables_initializer()
    sess.run(init)
    X_array = np.array([[0.4,0.2,0.4]])
    (_b,_W,_X,_y)=sess.run((b,W,X,y),feed_dict={X:X_array})
    print('b:')
    print(_b)    
    print('W:')
    print(_W)
    print('X:')
    print(_X)
    print('y:')
    print(_y)
b:
[[ 0.7134645 -0.5354284]]
W:
[[-0.58850515  1.0112841 ]
 [-0.5663574   0.25543067]
 [-0.31829494  0.5398116 ]]
X:
[[0.4 0.2 0.4]]
y:
[[0.23747298 0.136096  ]]
# 传入3*3数组

W = tf.Variable(tf.random_normal([3, 2]))
b = tf.Variable(tf.random_normal([1, 2]))
X = tf.placeholder("float", [None,3])
y=tf.nn.relu(tf.matmul(X,W)+b)
with tf.Session() as sess:
    init = tf.global_variables_initializer()
    sess.run(init)
    X_array = np.array([[0.4,0.2 ,0.4],
                        [0.3,0.4 ,0.5],
                        [0.3,-0.4,0.5]])    
    (_b,_W,_X,_y)=sess.run((b,W,X,y),feed_dict={X:X_array})
    print('b:')
    print(_b)    
    print('W:')
    print(_W)
    print('X:')
    print(_X)
    print('y:')
    print(_y)
b:
[[-3.3513153   0.38229984]]
W:
[[ 0.08664876  0.24946757]
 [-0.91662043 -2.2193696 ]
 [ 1.6511146  -0.34874526]]
X:
[[ 0.4  0.2  0.4]
 [ 0.3  0.4  0.5]
 [ 0.3 -0.4  0.5]]
y:
[[-0\.        -0\.       ]
 [-0\.        -0\.       ]
 [-0\.         1.1705153]]

创建layer函数以矩阵运算仿真神经网络

之前接触过Keras的多层感知器,现在我们要自己写感知器的层函数,为了后面更加的方便学习

  • output_dim :输出的神经元数量
  • input_dim : 输入的神经元数量
  • input :输入的二维数组placeholder
  • activation : 传入的激活函数默认None
# 定义layer函数,建立2层网络
# 以正态分布的随机数建立并初始化W(权重)
# 以正态分布的随机数建立b(偏差)
def layer(output_dim,input_dim,inputs, activation=None):
    W = tf.Variable(tf.random_normal([input_dim, output_dim]))
    b = tf.Variable(tf.random_normal([1, output_dim]))
    XWb = tf.matmul(inputs, W) + b
    if activation is None:
        outputs = XWb
    else:
        outputs = activation(XWb)
    return outputs
# 使用已建好的layer()函数建立3层神经网络
# 建立输入层X 输入神经元为4
# 建立隐藏层h 隐藏层神经元个数为4inputs = X 传入层
# activation = tf.nn.relu 定义激活函数relu

X = tf.placeholder("float", [None,4])
h = layer(output_dim=3 , input_dim=4 , inputs=X,activation=tf.nn.relu)
y = layer(output_dim=2,input_dim=3,inputs=h)
with tf.Session() as sess:
    init = tf.global_variables_initializer()
    sess.run(init)
    X_array = np.array([[0.4 , 0.2 ,0.4, 0.5]])
    (layer_X,layer_h , layer_y) = sess.run((X,h,y),feed_dict={X:X_array})
    print('input Layer X:')
    print(layer_X)
    print('hidden Layer h:')
    print(layer_h)
    print('output Layer y:')
    print(layer_y)
input Layer X:
[[0.4 0.2 0.4 0.5]]
hidden Layer h:
[[ 3.6888413 -0\.         0.8097873]]
output Layer y:
[[2.9887164 2.0248315]]
# 创建layer_debug()函数显示权重与偏差
def layer_debug(output_dim,input_dim,inputs, activation=None):
    W = tf.Variable(tf.random_normal([input_dim, output_dim]))
    b = tf.Variable(tf.random_normal([1, output_dim]))
    XWb = tf.matmul(inputs, W) + b
    if activation is None:
        outputs = XWb
    else:
        outputs = activation(XWb)
    return outputs,W,b
# 使用layer_debug()创建3层类神经网络
X = tf.placeholder("float", [None,4])
h,W1,b1=layer_debug(output_dim=3,input_dim=4,inputs=X,
                    activation=tf.nn.relu)
y,W2,b2=layer_debug(output_dim=2,input_dim=3,inputs=h)
with tf.Session() as sess:
    init = tf.global_variables_initializer()
    sess.run(init)
    X_array = np.array([[0.4,0.2 ,0.4,0.5]])    
    (layer_X,layer_h,layer_y,W1,b1,W2,b2)= \
             sess.run((X,h,y,W1,b1,W2,b2),feed_dict={X:X_array})
    print('input Layer X:')
    print(layer_X)
    print('W1:')
    print(  W1)    
    print('b1:')
    print(  b1)    
    print('hidden Layer h:')
    print(layer_h)    
    print('W2:')
    print(  W2)    
    print('b2:')
    print(  b2)    
    print('output Layer y:')
    print(layer_y)
input Layer X:
[[0.4 0.2 0.4 0.5]]
W1:
[[-0.3957269  -1.3121294   0.72043735]
 [-0.9788225   0.27537847  0.65960824]
 [-1.3185151   0.43064475  0.16570371]
 [-0.37439668 -0.9549402   0.08196367]]
b1:
[[-0.35466707 -0.5750693   0.7871804 ]]
hidden Layer h:
[[-0\.        -0\.         1.3145404]]
W2:
[[ 1.2650555   0.61335236]
 [-0.91212946 -0.63616544]
 [-0.48534423 -0.271578  ]]
b2:
[[-0.33160838  0.27007413]]
output Layer y:
[[-0.96961296 -0.08692613]]

版权声明:如无特殊说明,文章均为本站原创,转载请注明出处

本文链接:http://tunm.top/article/tf_2/