Caffe Faster - RCNN training VOC2007 no convergence problem-CodePudding

Windows platform, use the official faster - RCNN model (https://github.com/rbgirshick/py-faster-rcnn), which USES the end2end training method, all parameters are to keep the original file, from the very beginning training VOC2007, convergence, but the model has been unable to solve!

Super parameters:
train_net: "models/pascal_voc/VGG16/faster_rcnn_end2end/train prototxt"
Base_lr: 0.001
Lr_policy: "step"
Gamma: 0.1
Stepsize: 50000
Display: 20
Average_loss: 100
1 # iter_size:
Momentum: 0.9
Weight_decay: 0.0005
# We disable standard caffe solver snapshotting and implement our own snapshot
# function
The snapshot: 0
# We still use the snapshot prefix, Mr
Snapshot_prefix: "vgg16_faster_rcnn
"Iter_size: 2

1 round of training, the vector set it to 0.001, training more than 30000 times, not convergence,
I0703 08:38:56. 423439 2436 solver. CPP: 33400, 228] Iteration loss=2.10222
I0703 08:38:56. 423439 2436 solver. CPP: 244] "Train" net output # 0: loss_bbox=0.74249 (* 1=0.74249 loss)
I0703 08:38:56. 423439 2436 solver. CPP: 244] "Train" net output # 1: loss_cls=1.46208 (* 1=1.46208 loss)
I0703 08:38:56. 423439 2436 solver. CPP: 244] "Train" net output # 2: rpn_cls_loss=0.150039 (* 1=0.150039 loss)
I0703 08:38:56. 423439 2436 solver. CPP: 244] "Train" net output # 3: rpn_loss_bbox=0.014713 (* 1=0.014713 loss)
2436 sgd_solver I0703 08:38:56. 423439. The CPP: 33400, 106] Iteration lr=0.001
I0703 08:39:26. 140456 2436 solver. CPP: 33420, 228] Iteration loss=0.706823
I0703 08:39:26. 140456 2436 solver. CPP: 244] "Train" net output # 0: loss_bbox=0.154737 (* 1=0.154737 loss)
I0703 08:39:26. 140456 2436 solver. CPP: 244] "Train" net output # 1: loss_cls=0.421112 (* 1=0.421112 loss)
I0703 08:39:26. 140456 2436 solver. CPP: 244] "Train" net output # 2: rpn_cls_loss=0.08365 (* 1=0.08365 loss)
I0703 08:39:26. 140456 2436 solver. CPP: 244] "Train" net output # 3: rpn_loss_bbox=0.0374503 (* 1=0.0374503 loss)
2436 sgd_solver I0703 08:39:26. 140456. The CPP: 33420, 106] Iteration lr=0.001
I0703 08:39:55. 591863 2436 solver. CPP: 33440, 228] Iteration loss=1.00757
I0703 08:39:55. 591863 2436 solver. CPP: 244] "Train" net output # 0: loss_bbox=0.185621 (* 1=0.185621 loss)
I0703 08:39:55. 591863 2436 solver. CPP: 244] "Train" net output # 1: loss_cls=0.453262 (* 1=0.453262 loss)
I0703 08:39:55. 591863 2436 solver. CPP: 244] "Train" net output # 2: rpn_cls_loss=0.0994021 (* 1=0.0994021 loss)
I0703 08:39:55. 591863 2436 solver. CPP: 244] "Train" net output # 3: rpn_loss_bbox=0.00839208 (* 1=0.00839208 loss)
2436 sgd_solver I0703 08:39:55. 591863. The CPP: 33440, 106] Iteration lr=0.001
I0703 08:40:25. 324507 2436 solver. CPP: 33460, 228] Iteration loss=1.06348
I0703 08:40:25. 324507 2436 solver. CPP: 244] "Train" net output # 0: loss_bbox=0.220197 (* 1=0.220197 loss)
I0703 08:40:25. 324507 2436 solver. CPP: 244] "Train" net output # 1: loss_cls=0.700779 (* 1=0.700779 loss)
I0703 08:40:25. 324507 2436 solver. CPP: 244] "Train" net output # 2: rpn_cls_loss=0.110184 (* 1=0.110184 loss)
I0703 08:40:25. 324507 2436 solver. CPP: 244] "Train" net output # 3: rpn_loss_bbox=0.0254193 (* 1=0.0254193 loss)
2436 sgd_solver I0703 08:40:25. 324507. The CPP: 33460, 106] Iteration lr=0.001
I0703 08:40:55. 565562 2436 solver. CPP: 33480, 228] Iteration loss=0.994694
I0703 08:40:55. 565562 2436 solver. CPP: 244] "Train" net output # 0: loss_bbox=0.0659276 (* 1=0.0659276 loss)
I0703 08:40:55. 565562 2436 solver. CPP: 244] "Train" net output # 1: loss_cls=0.322046 (* 1=0.322046 loss)
I0703 08:40:55. 565562 2436 solver. CPP: 244] "Train" net output # 2: rpn_cls_loss=

2 times training, in the preservation of the first training model (30000 iterations) began training, on the basis of vector set as 0.0001, training more than ten thousand times again, still no convergence
I0703 14:27:09. 663796 2368 solver. CPP: 12980, 228] Iteration loss=0.946546
I0703 14:27:09. 663796 2368 solver. CPP: 244] "Train" net output # 0: loss_bbox=0.181305 (* 1=0.181305 loss)
I0703 14:27:09. 663796 2368 solver. CPP: 244] "Train" net output # 1: loss_cls=0.422951 (* 1=0.422951 loss)
I0703 14:27:09. 663796 2368 solver. CPP: 244] "Train" net output # 2: rpn_cls_loss=0.0988763 (* 1=0.0988763 loss)
I0703 14:27:09. 663796 2368 solver. CPP: 244] "Train" net output # 3: rpn_loss_bbox=0.173789 (* 1=0.173789 loss)
2368 sgd_solver I0703 14:27:09. 663796. The CPP: 12980, 106] Iteration lr=0.0001
I0703 14:27:40. 124526 2368 solver. CPP: 13000, 228] Iteration loss=1.26704
I0703 14:27:40. 124526 2368 solver. CPP: 244] "Train" net output # 0: loss_bbox=0.30089 (* 1=0.30089 loss)
I0703 14:27:40. 124526 2368 solver. CPP: 244] "Train" net output # 1: loss_cls=0.715131 (* 1=0.715131 loss)
I0703 14:27:40. 125524 2368 solver. CPP: 244] "Train" net output # 2: rpn_cls_loss=0.139781 (* 1=0.139781 loss)
nullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnull