自动驾驶-标记车道2
发布时间
阅读量：
阅读量
为了确保无人驾驶汽车能够在公路上安全行驶,它必须遵循与传统车辆相同的交通法规.其中最基础的规定是车辆必须沿着车道线平稳前行.无人驾驶汽车依靠摄像头捕捉前方道路图像,然后通过对这些图像进行一系列处理来确定前方车道的位置,从而明确行车路径 boundaries.前面章节中介绍了一种名为candy-based edge detection method的技术,用于标记车道线,但该方法的一个局限性是仅适用于直线场景,无法应对曲线或弯道的情况.本章将介绍一种更加健壮的解决方案来改进这一问题
本章给出的标记车道的方法可以概述为以下几步：
去除图像变形；
采用Sobel算子检测并提取主车道线；
将原始图像转换为俯视图；
对探测到的道路边界进行多项式拟合，并计算其曲率以获得平滑的道路模型；
通过逆变换过程获得标定的道路边界；
下面描述每一步的内容并给出源码和输出
消除图像畸变
成像过程将三维空间的影像转化为二维平面，并伴随几何畸变现象的发生。在实际应用中，若直接利用畸变后的图像提取车道线，则可能导致误判车道曲线程度，并进而影响计算出的道路曲线半径值。因此，在第一步中必须先校正相机参数以消除几何畸变。具体来说，在该方法中首先提供多幅图像作为输入样本，并基于国际象棋棋盘这一具有明显特征的物体进行分析。由于国际象棋棋盘具有黑白格纹分明且对比度高的特点，在此场景下能够较为容易地识别出标准物 mark 的位置信息并完成定位操作。通过分析棋盘格纹交点位置信息确定精确坐标位置，并在此基础上构建一个与原始交点列表规模相当的新列表以记录这些关键点的位置数据。随后通过计算原始交点序列与新构建序列之间的差异性来推导出相机镜头所引入的几何畸变参数值，并基于这些参数值对原始图像进行校正处理以消除其造成的形变影响。
找到棋盘交点
import numpy as np
import cv2
import glob
import matplotlib.pyplot as plt
#%matplotlib qt
first_frame = 1
# prepare object points, like (0,0,0), (1,0,0), (2,0,0) ....,(6,5,0)
objp = np.zeros((6*9,3), np.float32)
objp[:,:2] = np.mgrid[0:9, 0:6].T.reshape(-1,2)
# Arrays to store object points and image points from all the images.
objpoints = [] # 3d points in real world space
imgpoints = [] # 2d points in image plane.
# Make a list of calibration images
images = glob.glob('camera_cal/cali*.jpg')
sorted(images)
# Step through the list and search for chessboard corners
for idx, fname in enumerate(images):
img = cv2.imread(fname)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# Find the chessboard corners
ret, corners = cv2.findChessboardCorners(gray, (9,6), None)
# If found, add object points, image points
if ret == True:
objpoints.append(objp)
imgpoints.append(corners)
# Draw and display the corners
cv2.drawChessboardCorners(img, (9,6), corners, ret)
write_name = 'camera_cal/corners_found'+str(idx)+'.jpg'
cv2.imwrite(write_name, img)
img = cv2.imread('camera_cal/calibration3.jpg')
dst = cv2.imread("camera_cal/corners_found4.jpg")
f, (ax1, ax2) = plt.subplots(1, 2, figsize=(20,10))
ax1.imshow(img)
ax1.set_title('Original Image', fontsize=30)
ax2.imshow(dst)
ax2.set_title('Find Corner Image', fontsize=30)
消除畸变
import pickle
%matplotlib inline
# Test undistortion on an image
img = cv2.imread('camera_cal/calibration1.jpg')
def undistort_image(img,objpoints,imgpoints):
# Do camera calibration given object points and image points
#def undistort_image(img,objpoints,imgpoints):
img_size = (img.shape[1], img.shape[0])
ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(objpoints, imgpoints, img_size,None,None)
dst = cv2.undistort(img, mtx, dist, None, mtx)
return mtx,dist,dst
mtx,dist,dst = undistort_image(img,objpoints,imgpoints)
cv2.imwrite('camera_cal/test_undist.jpg',dst)
# Save the camera calibration result for later use (we won't worry about rvecs / tvecs)
dist_pickle = {}
dist_pickle["mtx"] = mtx
dist_pickle["dist"] = dist
pickle.dump( dist_pickle, open( "wide_dist_pickle.p", "wb" ) )
#dst = cv2.cvtColor(dst, cv2.COLOR_BGR2RGB)
# Visualize undistortion
f, (ax1, ax2) = plt.subplots(1, 2, figsize=(20,10))
ax1.imshow(img)
ax1.set_title('Original Image', fontsize=30)
ax2.imshow(dst)
ax2.set_title('Undistorted Image', fontsize=30)
对道路消除畸变
用sobel算子提取车道
在消除图片畸变之后，接下来的操作是对图像进行处理以提取车道信息。sobel算子通过计算像素在特定方向上的梯度变化来识别边缘。当该方向上的相邻像素对比度较高时，则该方向上的梯度变化幅度较大。观察上图可知，在y轴方向上具有明显的特征，在此基础之上应用sobel算子并对其x轴方向进行梯度计算；随后将计算结果中小于设定阈值的部分予以抑制或滤除。
提取车道
import numpy as np
import cv2
import matplotlib.pyplot as plt
# Edit this function to create your own pipeline.
def binary_image(img, s_thresh=(170, 255), sx_thresh=(20, 100)):
img = np.copy(img)
# Convert to HSV color space and separate the V channel
hsv = cv2.cvtColor(img, cv2.COLOR_RGB2HLS).astype(np.float)
l_channel = hsv[:,:,1]
s_channel = hsv[:,:,2]
# Sobel x
sobelx = cv2.Sobel(l_channel, cv2.CV_64F, 1, 0) # Take the derivative in x
abs_sobelx = np.absolute(sobelx) # Absolute x derivative to accentuate lines away from horizontal
scaled_sobel = np.uint8(255*abs_sobelx/np.max(abs_sobelx))
# Threshold x gradient
sxbinary = np.zeros_like(scaled_sobel)
sxbinary[(scaled_sobel >= sx_thresh[0]) & (scaled_sobel <= sx_thresh[1])] = 1
# Threshold color channel
s_binary = np.zeros_like(s_channel)
s_binary[(s_channel >= s_thresh[0]) & (s_channel <= s_thresh[1])] = 1
# Stack each channel
# Note color_binary[:, :, 0] is all 0s, effectively an all black image. It might
# be beneficial to replace this channel with something else.
color_binary = np.dstack(( np.zeros_like(sxbinary), sxbinary, s_binary))
# Combine the two binary thresholds
combined_binary = np.zeros_like(sxbinary)
combined_binary[(s_binary == 1) | (sxbinary == 1)] = 1
#return color_binary
return combined_binary
binary = binary_image(undist)
# Plot the result
f, (ax1, ax2) = plt.subplots(1, 2, figsize=(24, 9))
f.tight_layout()
ax1.imshow(img)
ax1.set_title('Original Image', fontsize=40)
ax2.imshow(binary)
ax2.set_title('Pipeline Result', fontsize=40)
plt.subplots_adjust(left=0., right=1, top=0.9, bottom=0.)
转换为鸟瞰图
通过上一步骤处理后可以获得更加清晰地呈现的车道边界线；然而由于透视效果的原因，在距离镜头较近的位置 lane markings 显示得更大而在远处则显得较小 因此车辆两侧原本平行的道路边缘在线条交汇处看上去似乎是相交的 但这只是视觉上的错觉 实际上它们始终是相互平行的 为了精确计算车道曲线 所需的方法是将图像转换为俯视图姿势
import pickle
import cv2
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
# Read in the saved camera matrix and distortion coefficients
# These are the arrays you calculated using cv2.calibrateCamera()
dist_pickle = pickle.load( open( "wide_dist_pickle.p", "rb" ) )
mtx = dist_pickle["mtx"]
dist = dist_pickle["dist"]
# Read in an image
#img = mpimg.imread('test1.jpg')
#nx = 9 # the number of inside corners in x
#ny = 6 # the number of inside corners in y
# Define a function that takes an image, number of x and y points,
# camera matrix and distortion coefficients
# camera matrix and distortion coefficients
def binary_unwarp(img, mtx, dist):
# Use the OpenCV undistort() function to remove distortion
#undist = cv2.undistort(img, mtx, dist, None, mtx)
# Convert undistorted image to grayscale
#gray = cv2.cvtColor(undist, cv2.COLOR_BGR2GRAY)
# Search for corners in the grayscaled image
#ret, corners = cv2.findChessboardCorners(gray, (nx, ny), None)
#if ret == True:
# If we found corners, draw them! (just for fun)
#cv2.drawChessboardCorners(undist, (nx, ny), corners, ret)
# Choose offset from image corners to plot detected corners
# This should be chosen to present the result at the proper aspect ratio
# My choice of 100 pixels is not exact, but close enough for our purpose here
offset = 100 # offset for dst points
# Grab the image shape
img_size = (img.shape[1], img.shape[0])
# For source points I'm grabbing the outer four detected corners
#src = np.float32([[850,320], [865,450], [533,350], [535,210]])
src = np.float32([[298,683], [563,479], [753,478], [1082,673]])
# For destination points, I'm arbitrarily choosing some points to be
# a nice fit for displaying our warped result
# again, not exact, but close enough for our purposes
#dst = np.float32([ [1150,450], [1150,700],[200, 700], [200, 450]])
dst = np.float32([src[0], [src[0][0],200],[src[3][0], 200], src[3]])
#dst = np.float32([[img_size[0]-offset, offset],
# [img_size[0]-offset, img_size[1]-offset],
# [offset, img_size[1]-offset],
# [offset, offset]])
# Given src and dst points, calculate the perspective transform matrix
M = cv2.getPerspectiveTransform(src, dst)
# Warp the image using OpenCV warpPerspective()
Minv = cv2.getPerspectiveTransform(dst, src)
warped = cv2.warpPerspective(img, M, img_size)
# Return the resulting image and matrix
return warped, M,Minv
binary_warped, perspective_M,Minv = binary_unwarp(binary, mtx, dist)
f, (ax1, ax2) = plt.subplots(1, 2, figsize=(24, 9))
f.tight_layout()
ax1.imshow(binary)
ax1.set_title('Original Image', fontsize=50)
ax2.imshow(binary_warped)
ax2.set_title('Undistorted and Warped Image', fontsize=50)
plt.subplots_adjust(left=0., right=1, top=0.9, bottom=0.)
对车道多项式拟合，计算曲率
首先将俯视图转换为直方图，并沿垂直方向将其划分为九个区域。对于每个区域而言，在其对应的直方图中找到峰值位置并确定左右车道线中间的位置。接下来通过多项式拟合方法获得左右车道线的具体坐标。最后评估车道线轨迹的弯曲程度
import numpy as np
import cv2
import matplotlib.pyplot as plt
margin = 100
# Set minimum number of pixels found to recenter window
minpix = 50
def calculate_curverad(leftx,lefty,rightx,righty,ploty,image_size):
y_eval = np.max(ploty)
# Define conversions in x and y from pixels space to meters
ym_per_pix = 30/720 # meters per pixel in y dimension
xm_per_pix = 3.7/700 # meters per pixel in x dimension
# Fit new polynomials to x,y in world space
left_fit_cr = np.polyfit(lefty*ym_per_pix, leftx*xm_per_pix, 2)
right_fit_cr = np.polyfit(righty*ym_per_pix, rightx*xm_per_pix, 2)
# Calculate the new radii of curvature
left_curverad = ((1 + (2*left_fit_cr[0]*y_eval*ym_per_pix + left_fit_cr[1])**2)**1.5) / np.absolute(2*left_fit_cr[0])
right_curverad = ((1 + (2*right_fit_cr[0]*y_eval*ym_per_pix + right_fit_cr[1])**2)**1.5) / np.absolute(2*right_fit_cr[0])
# Now our radius of curvature is in meters
scene_height = image_size[0]*ym_per_pix
scene_width = image_size[1]*xm_per_pix
left_intercept = left_fit_cr[0]*scene_height**2+left_fit_cr[1]*scene_height+left_fit_cr[2]
right_intercept = right_fit_cr[0]*scene_height**2+right_fit_cr[1]*scene_height+right_fit_cr[2]
calculated_center = (left_intercept+right_intercept)/2.0
center_offset = (calculated_center - scene_width/2.0)
return left_curverad,right_curverad,center_offset # Create empty lists to receive left and right lane pixel indices
def get_polynomial_first(binary_warped):
# Assuming you have created a warped binary image called "binary_warped"
# Take a histogram of the bottom half of the image
histogram = np.sum(binary_warped[binary_warped.shape[0]//2:,:], axis=0)
# Create an output image to draw on and visualize the result
out_img = np.dstack((binary_warped, binary_warped, binary_warped))*255
# Find the peak of the left and right halves of the histogram
# These will be the starting point for the left and right lines
midpoint = np.int(histogram.shape[0]//2)
leftx_base = np.argmax(histogram[:midpoint])
rightx_base = np.argmax(histogram[midpoint:]) + midpoint
# Choose the number of sliding windows
nwindows = 9
# Set height of windows
window_height = np.int(binary_warped.shape[0]/nwindows)
# Identify the x and y positions of all nonzero pixels in the image
nonzero = binary_warped.nonzero()
nonzeroy = np.array(nonzero[0])
nonzerox = np.array(nonzero[1])
# Current positions to be updated for each window
leftx_current = leftx_base
rightx_current = rightx_base
# Set the width of the windows +/- margin
left_lane_inds = []
right_lane_inds = []
# Step through the windows one by one
for window in range(nwindows):
# Identify window boundaries in x and y (and right and left)
win_y_low = binary_warped.shape[0] - (window+1)*window_height
win_y_high = binary_warped.shape[0] - window*window_height
win_xleft_low = leftx_current - margin
win_xleft_high = leftx_current + margin
win_xright_low = rightx_current - margin
win_xright_high = rightx_current + margin
# Draw the windows on the visualization image
cv2.rectangle(out_img,(win_xleft_low,win_y_low),(win_xleft_high,win_y_high),(0,255,0), 2)
cv2.rectangle(out_img,(win_xright_low,win_y_low),(win_xright_high,win_y_high),(0,255,0), 2)
# Identify the nonzero pixels in x and y within the window
good_left_inds = ((nonzeroy >= win_y_low) & (nonzeroy < win_y_high) & (nonzerox >= win_xleft_low) & (nonzerox < win_xleft_high)).nonzero()[0]
good_right_inds = ((nonzeroy >= win_y_low) & (nonzeroy < win_y_high) & (nonzerox >= win_xright_low) & (nonzerox < win_xright_high)).nonzero()[0]
# Append these indices to the lists
left_lane_inds.append(good_left_inds)
right_lane_inds.append(good_right_inds)
# If you found > minpix pixels, recenter next window on their mean position
if len(good_left_inds) > minpix:
leftx_current = np.int(np.mean(nonzerox[good_left_inds]))
if len(good_right_inds) > minpix:
rightx_current = np.int(np.mean(nonzerox[good_right_inds]))
# Concatenate the arrays of indices
left_lane_inds = np.concatenate(left_lane_inds)
right_lane_inds = np.concatenate(right_lane_inds)
# Extract left and right line pixel positions
leftx = nonzerox[left_lane_inds]
lefty = nonzeroy[left_lane_inds]
rightx = nonzerox[right_lane_inds]
righty = nonzeroy[right_lane_inds]
# Fit a second order polynomial to each
left_fit = np.polyfit(lefty, leftx, 2)
right_fit = np.polyfit(righty, rightx, 2)
# Generate x and y values for plotting
ploty = np.linspace(0, binary_warped.shape[0]-1, binary_warped.shape[0] )
left_fitx = left_fit[0]*ploty**2 + left_fit[1]*ploty + left_fit[2]
right_fitx = right_fit[0]*ploty**2 + right_fit[1]*ploty + right_fit[2]
out_img[nonzeroy[left_lane_inds], nonzerox[left_lane_inds]] = [255, 0, 0]
out_img[nonzeroy[right_lane_inds], nonzerox[right_lane_inds]] = [0, 0, 255]
l,r,d = calculate_curverad(leftx,lefty,rightx,righty,ploty,binary_warped.shape)
return left_fitx,right_fitx,ploty,left_fit,right_fit,l,r,d,out_img
left_fitx,right_fitx,ploty,left_fit,right_fit,left_radius,right_radius,center_offset,out_img = get_polynomial_first(binary_warped)
plt.imshow(out_img)
plt.plot(left_fitx, ploty, color='yellow')
plt.plot(right_fitx, ploty, color='yellow')
plt.xlim(0, 1280)
plt.ylim(720, 0)
在原始图像中标记车道
在找到车道的过程中，在逆变换的作用下完成了鸟瞰图的还原；经过叠加处理后与原始图像结合形成了标记的车道线。
left_fit_next=[]
right_fit_next=[]
def draw_lane(binary_warped,undist,Minv,
left_fitx,right_fitx,ploty,left_radius,right_radius,center_offset):
# Create an image to draw the lines on
warp_zero = np.zeros_like(binary_warped).astype(np.uint8)
color_warp = np.dstack((warp_zero, warp_zero, warp_zero))
# Recast the x and y points into usable format for cv2.fillPoly()
pts_left = np.array([np.transpose(np.vstack([left_fitx, ploty]))])
pts_right = np.array([np.flipud(np.transpose(np.vstack([right_fitx, ploty])))])
pts = np.hstack((pts_left, pts_right))
# Draw the lane onto the warped blank image
cv2.fillPoly(color_warp, np.int_([pts]), (0,255, 0))
# Warp the blank back to original image space using inverse perspective matrix (Minv)
newwarp = cv2.warpPerspective(color_warp, Minv, (undist.shape[1], undist.shape[0]))
# Combine the result with the original image
result = cv2.addWeighted(undist, 1, newwarp, 0.3, 0)
font = cv2.FONT_HERSHEY_SIMPLEX
curvature = 0.5*(round(right_radius/1000,1) + round(left_radius/1000,1))
str1 = str('Radius of Curtature: '+str(curvature)+'km')
cv2.putText(result,str1,(30,60),font,1,(255,255,255),2)
str2 = str('Vehicle is '+str(np.round(center_offset,2))+' m ' +'left of center')
cv2.putText(result,str2,(30,90),font,1,(255,255,255),2)
return result, newwarp, color_warp
result, newwarp, color_warp = draw_lane(binary_warped,undist,
Minv,left_fitx,right_fitx,ploty,left_radius,
right_radius,center_offset)
plt.imshow(result)
以上是udacity 自动驾驶课程第一学期第四个项目的总结。
全部评论 (0)
还没有任何评论哟~