Repository navigation
Expand file tree
/
Copy pathatom.xml
More file actions
2339 lines (2128 loc) · 300 KB
/
Copy pathatom.xml
File metadata and controls
2339 lines (2128 loc) · 300 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
<title>李飞阳</title>
<subtitle>PM、Coder、Data mining</subtitle>
<link href="/atom.xml" rel="self"/>
<link href="http://feiyang.li/"/>
<updated>2017-06-08T04:10:47.347Z</updated>
<id>http://feiyang.li/</id>
<author>
<name>Wing Lee</name>
</author>
<generator uri="http://hexo.io/">Hexo</generator>
<entry>
<title>算法实验(二)</title>
<link href="http://feiyang.li/2017/06/08/algorithm2/index.html"/>
<id>http://feiyang.li/2017/06/08/algorithm2/index.html</id>
<published>2017-06-08T04:09:41.000Z</published>
<updated>2017-06-08T04:10:47.347Z</updated>
<content type="html"><![CDATA[<script src="/assets/js/DPlayer.min.js"> </script><script src="/assets/js/APlayer.min.js"> </script><blockquote>
<p>任务调度问题:在单处理器上具有期限和惩罚的单位时间任务调度问题;平衡树问题:实现3种树中的两种:红黑树,AVL树,Treap树。</p>
</blockquote>
<hr>
<a id="more"></a>
<h1 id="任务调度问题"><a href="#任务调度问题" class="headerlink" title="任务调度问题"></a>任务调度问题</h1><p>任务调度问题就是给定一个有穷单位时间任务的集合S,集合S中的每个任务都有一个截止期限di和超时惩罚wi,需要找出集合S的一个调度,使得因任务误期所导致的总惩罚最小,这个调度也称为S的一个最优调度。</p>
<h2 id="问题描述:"><a href="#问题描述:" class="headerlink" title="问题描述:"></a>问题描述:</h2><p>在单处理器上具有期限和惩罚的单位时间任务调度问题(课本P239)<br>实验要求:<br>(1)实现这个问题的贪心算法<br>(2)将每个 wi 替换为max{m1,m2…mn}—wi,运行算法比较结果。 </p>
<h2 id="解题思路:"><a href="#解题思路:" class="headerlink" title="解题思路:"></a>解题思路:</h2><p>1.先将任务按照时间惩罚递减顺序进行排序,<br>2.然后用贪心的思想,尽量把惩罚重的任务先放入待完成队列中。 </p>
<h2 id="代码"><a href="#代码" class="headerlink" title="代码"></a>代码</h2><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div><div class="line">35</div><div class="line">36</div><div class="line">37</div><div class="line">38</div><div class="line">39</div><div class="line">40</div><div class="line">41</div><div class="line">42</div><div class="line">43</div><div class="line">44</div><div class="line">45</div><div class="line">46</div><div class="line">47</div><div class="line">48</div><div class="line">49</div><div class="line">50</div><div class="line">51</div><div class="line">52</div><div class="line">53</div><div class="line">54</div><div class="line">55</div><div class="line">56</div><div class="line">57</div><div class="line">58</div><div class="line">59</div><div class="line">60</div><div class="line">61</div><div class="line">62</div><div class="line">63</div><div class="line">64</div><div class="line">65</div><div class="line">66</div><div class="line">67</div><div class="line">68</div><div class="line">69</div><div class="line">70</div><div class="line">71</div><div class="line">72</div><div class="line">73</div><div class="line">74</div><div class="line">75</div><div class="line">76</div><div class="line">77</div><div class="line">78</div><div class="line">79</div><div class="line">80</div><div class="line">81</div><div class="line">82</div><div class="line">83</div><div class="line">84</div><div class="line">85</div><div class="line">86</div><div class="line">87</div><div class="line">88</div><div class="line">89</div><div class="line">90</div><div class="line">91</div><div class="line">92</div><div class="line">93</div><div class="line">94</div><div class="line">95</div><div class="line">96</div><div class="line">97</div><div class="line">98</div><div class="line">99</div><div class="line">100</div><div class="line">101</div><div class="line">102</div></pre></td><td class="code"><pre><div class="line">#include<iostream></div><div class="line">#include <algorithm></div><div class="line">#include<ctime></div><div class="line">#define n 6</div><div class="line">using namespace std;</div><div class="line">typedef struct {</div><div class="line"> int id;//第几个任务</div><div class="line"> int d;//任务的期限</div><div class="line"> int w;//罚</div><div class="line">}tesk;</div><div class="line"></div><div class="line">bool cmpW(tesk t1, tesk t2) {</div><div class="line"> return t1.w>t2.w;</div><div class="line">}</div><div class="line">bool cmpD(tesk t1, tesk t2) {</div><div class="line"> return t1.d<t2.d;</div><div class="line">}</div><div class="line">void InitNum(tesk t[]) {</div><div class="line"> srand((unsigned)time(NULL));</div><div class="line"> for (int i = 0; i < n; i++) {</div><div class="line"> t[i].id = i + 1;</div><div class="line"> t[i].d = rand() % n + 1;</div><div class="line"> t[i].w = rand() % 30;</div><div class="line"> }</div><div class="line">}</div><div class="line"></div><div class="line">void copy(tesk &t1, tesk &t2) {</div><div class="line"> t1.id = t2.id;</div><div class="line"> t1.d = t2.d;</div><div class="line"> t1.w = t2.w;</div><div class="line">}</div><div class="line"></div><div class="line">int greedy(tesk a[], tesk ta[], int &ans) {</div><div class="line"> int num = 0, i, j;//当前已经完成的任务数量</div><div class="line"> sort(ta, ta + n, cmpW);</div><div class="line"> int fla[9999];</div><div class="line"> memset(fla, 0, sizeof(fla));</div><div class="line"> for (i = 0; i<n; i++) {</div><div class="line"> for (j = ta[i].d; j>0; j--) {</div><div class="line"> if (fla[j] == 0) {</div><div class="line"> fla[j] = 1;</div><div class="line"> break;</div><div class="line"> }</div><div class="line"> }</div><div class="line"> if (j>0) {</div><div class="line"> copy(a[num++], ta[i]);</div><div class="line"> ans -= ta[i].w;</div><div class="line"> }</div><div class="line"> }</div><div class="line"> return num;</div><div class="line">}</div><div class="line"></div><div class="line"></div><div class="line">int main() {</div><div class="line"> tesk ta[n], a[n], tb[n], b[n];</div><div class="line"> printf("%s", "正在生成随机数据\n");</div><div class="line"> InitNum(ta);</div><div class="line"> for (int i = 0; i < n; i++) {</div><div class="line"> copy(tb[i], ta[i]);</div><div class="line"> }</div><div class="line"> printf("%s", "生成的数据为\n");</div><div class="line"> int max_w = -1, sum_w = 0;</div><div class="line"> for (int i = 0; i < n; i++) {</div><div class="line"> printf("ID: %d,任务的期限: %d,任务的惩罚:%d\n", ta[i].id, ta[i].d, ta[i].w);</div><div class="line"> sum_w += ta[i].w;</div><div class="line"> if (max_w < ta[i].w) {</div><div class="line"> max_w = ta[i].w;</div><div class="line"> }</div><div class="line"> }</div><div class="line"> int k = greedy(a, ta, sum_w);</div><div class="line"> printf("任务惩罚为:%d\n", sum_w);</div><div class="line"> sort(a, a + k, cmpD);</div><div class="line"> printf("将完成执行的任务按照时间递增顺序排列输出:\n");</div><div class="line"> for (int i = 0; i<k; i++) {</div><div class="line"> printf("ID:%d,期限:%d,惩罚:%d\n", a[i].id, a[i].d, a[i].w);</div><div class="line"> }</div><div class="line"> sort(a, a + k, cmpW);</div><div class="line"> printf("将完成的任务按惩罚排序;\n");</div><div class="line"> for (int i = 0; i<k; i++) {</div><div class="line"> printf("ID:%d,期限:%d,惩罚:%d\n", a[i].id, a[i].d, a[i].w);</div><div class="line"> }</div><div class="line"></div><div class="line"> printf("若将每个wi替换为max{m1,m2...mn}—wi,新的数据为\n");</div><div class="line"> for (int i = 0; i<n; i++) {</div><div class="line"> tb[i].w = max_w - tb[i].w;</div><div class="line"> printf("ID:%d,期限:%d,惩罚:%d\n", tb[i].id, tb[i].d, tb[i].w);</div><div class="line"> sum_w += tb[i].w;</div><div class="line"> }</div><div class="line"> k = greedy(b, tb, sum_w);</div><div class="line"> printf("任务惩罚为:%d\n", sum_w);</div><div class="line"> sort(b, b + k, cmpD);</div><div class="line"> printf("将完成执行的任务按照时间递增顺序排列输出:\n");</div><div class="line"> for (int i = 0; i<k; i++) {</div><div class="line"> printf("ID:%d,期限:%d,惩罚:%d\n", b[i].id, b[i].d, b[i].w);</div><div class="line"> }</div><div class="line"> sort(b, b + k, cmpW);</div><div class="line"> printf("将完成的任务按惩罚排序;\n");</div><div class="line"> for (int i = 0; i<k; i++) {</div><div class="line"> printf("ID:%d,期限:%d,惩罚:%d\n", b[i].id, b[i].d, b[i].w);</div><div class="line"> }</div><div class="line"></div><div class="line">}</div></pre></td></tr></table></figure>
<h2 id="实验截图"><a href="#实验截图" class="headerlink" title="实验截图"></a>实验截图</h2><figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://cdn.mmmxcc.cn/blog/20170605/190729182.png?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="" title="">
</div>
<div class="image-caption"></div>
</figure>
<h1 id="实现3种树"><a href="#实现3种树" class="headerlink" title="实现3种树"></a>实现3种树</h1><p>实现3种树中的两种:红黑树,AVL树(课本P177),Treap树</p>
<h2 id="红黑树"><a href="#红黑树" class="headerlink" title="红黑树"></a>红黑树</h2><h3 id="性质"><a href="#性质" class="headerlink" title="性质"></a>性质</h3><ul>
<li>根节点是黑色的</li>
<li>每个节点或红或黑</li>
<li>每个叶节点都是黑色的</li>
<li>红节点的两个子节点都是黑色</li>
</ul>
]]></content>
<summary type="html">
<blockquote>
<p>任务调度问题:在单处理器上具有期限和惩罚的单位时间任务调度问题;平衡树问题:实现3种树中的两种:红黑树,AVL树,Treap树。</p>
</blockquote>
<hr>
</summary>
<category term="Algorithms" scheme="http://feiyang.li/tags/Algorithms/"/>
<category term="cpp" scheme="http://feiyang.li/tags/cpp/"/>
</entry>
<entry>
<title>Python mysql表数据和json格式的相互转换</title>
<link href="http://feiyang.li/2017/05/31/python-mysql-json/index.html"/>
<id>http://feiyang.li/2017/05/31/python-mysql-json/index.html</id>
<published>2017-05-31T13:01:18.000Z</published>
<updated>2017-05-31T13:02:02.907Z</updated>
<content type="html"><![CDATA[<script src="/assets/js/DPlayer.min.js"> </script><script src="/assets/js/APlayer.min.js"> </script><blockquote>
<p>今天碰到了一个比较奇葩的需求,将mysql表数据转换成json格式。我顺便研究了一下,写了个python。</p>
</blockquote>
<hr>
<a id="more"></a>
<ul>
<li>数据库相关的<a href="http://feiyang.li/2017/05/16/databases/index.html">基本操作</a> </li>
<li>如果使用<a href="http://feiyang.li/2017/02/10/python_camp3/index.html">python操作mysql</a></li>
</ul>
<p>要我说,数据库操作学会了之后,会节省很多很多时间。</p>
<h1 id="SQL数据—-gt-json数据"><a href="#SQL数据—-gt-json数据" class="headerlink" title="SQL数据—>json数据"></a>SQL数据—>json数据</h1><figure class="highlight python"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div><div class="line">35</div><div class="line">36</div><div class="line">37</div><div class="line">38</div><div class="line">39</div><div class="line">40</div><div class="line">41</div><div class="line">42</div><div class="line">43</div><div class="line">44</div></pre></td><td class="code"><pre><div class="line"><span class="comment">#!/usr/bin/env python</span></div><div class="line"><span class="comment">#-*- coding: utf8 -*-</span></div><div class="line"></div><div class="line"><span class="keyword">import</span> MySQLdb</div><div class="line"><span class="keyword">import</span> json</div><div class="line"></div><div class="line"><span class="comment"># 定义查询SQL语句</span></div><div class="line">sql = <span class="string">"SELECT id,NAME,LOCAL,mobile,CreateTime FROM db1.s1;"</span></div><div class="line"></div><div class="line"><span class="comment"># 设置列的别名或者直接用表字段名:(下面的sql含有别名)</span></div><div class="line"></div><div class="line"><span class="comment"># sql="SELECT s.id AS 1_id,s.name AS 2_na,s.local AS 3_lo,s.mobile AS 4_mo,s.CreateTime AS 5_ct FROM db1.s1 s;"</span></div><div class="line"></div><div class="line"></div><div class="line"><span class="comment"># 定义连接MySQL的登录信息(此处以字典形式)</span></div><div class="line">Loginfo = {<span class="string">'USER'</span>:<span class="string">'admin'</span>, <span class="string">'PSWD'</span>:<span class="string">'admin'</span>, <span class="string">'HOST'</span>:<span class="string">'10.10.60.108'</span>, <span class="string">'PORT'</span>:<span class="number">4001</span>}</div><div class="line"></div><div class="line"></div><div class="line"><span class="comment"># Python 连接MySQL</span></div><div class="line">conn=MySQLdb.connect(host=Loginfo[<span class="string">'HOST'</span>],user=Loginfo[<span class="string">'USER'</span>],passwd=Loginfo[<span class="string">'PSWD'</span>],port=Loginfo[<span class="string">'PORT'</span>],charset=<span class="string">'utf8'</span>)</div><div class="line">cur=conn.cursor()</div><div class="line">cur.execute(sql) <span class="comment"># 执行SQL查询</span></div><div class="line">data = cur.fetchall() <span class="comment"># 查询结果给data。如果执行:print data 显示结果:((第一行内容),(第二行内容),(第三行内容),(第四行内容))</span></div><div class="line">fields = cur.description <span class="comment"># 获取查询结果中列的字段名,如果查询SQL中使用别名,此处显示别名。 </span></div><div class="line">cur.close()</div><div class="line">conn.close()</div><div class="line"></div><div class="line"><span class="comment"># Main </span></div><div class="line">column_list = [] <span class="comment"># 定义字段名的列表</span></div><div class="line"><span class="keyword">for</span> i <span class="keyword">in</span> fields:</div><div class="line"> column_list.append(i[<span class="number">0</span>]) <span class="comment"># 提取字段名,追加到列表中</span></div><div class="line"><span class="comment">#print column_list # 列表显示结果:['id', 'NAME', 'LOCAL', 'mobile', 'CreateTime']</span></div><div class="line"></div><div class="line"><span class="keyword">with</span> open(<span class="string">'/data/scripts/logdb/json.txt'</span>,<span class="string">'w+'</span>) <span class="keyword">as</span> f: <span class="comment"># 打开输出结果文件</span></div><div class="line"> <span class="keyword">for</span> row <span class="keyword">in</span> data: <span class="comment"># 一次循环,row代表一行,row以元组的形式显示。</span></div><div class="line"> result = {} <span class="comment"># 定义Python 字典</span></div><div class="line"> result[column_list[<span class="number">0</span>]] = row[<span class="number">0</span>] <span class="comment"># 将row中的每个元素,追加到字典中。 </span></div><div class="line"> result[column_list[<span class="number">1</span>]] = str(row[<span class="number">1</span>]) <span class="comment"># Python字段格式 和json字段格式转换</span></div><div class="line"> result[column_list[<span class="number">2</span>]] = str(row[<span class="number">2</span>]) </div><div class="line"> result[column_list[<span class="number">3</span>]] = row[<span class="number">3</span>]</div><div class="line"> result[column_list[<span class="number">4</span>]] = str(row[<span class="number">4</span>])</div><div class="line"> jsondata=json.dumps(result,ensure_ascii=<span class="keyword">False</span>) <span class="comment"># Python的dict --转换成----> json的object</span></div><div class="line"> f.write(jsondata + <span class="string">'\n'</span>) <span class="comment"># 写入文件</span></div><div class="line">f.close()</div></pre></td></tr></table></figure>
<p>注意:最后输出结果,列的顺序是随机的。</p>
<p> 如果想要顺序输出:1.字段起别名,比如:1_xx,2_xx,3_xx…..然后在在json.dumps中利用:sort_keys=True排序,结果就顺序输出。</p>
<p>例如:</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">print json.dumps({"c": 0, "b": 0, "a": 0}, sort_keys=True)</div></pre></td></tr></table></figure>
<p>{“a”: 0, “b”: 0, “c”: 0}</p>
<h1 id="json格式—-gt-sql语句"><a href="#json格式—-gt-sql语句" class="headerlink" title="json格式—>sql语句"></a>json格式—>sql语句</h1><figure class="highlight python"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div></pre></td><td class="code"><pre><div class="line"><span class="comment">#!/usr/bin/env python</span></div><div class="line"><span class="comment">#-*- coding: utf8 -*-</span></div><div class="line"></div><div class="line"><span class="keyword">import</span> MySQLdb</div><div class="line"><span class="keyword">import</span> json</div><div class="line"></div><div class="line">datalist = [] <span class="comment"># python 列表</span></div><div class="line"><span class="keyword">with</span> open(<span class="string">'/data/scripts/logdb/json.txt'</span>,<span class="string">'r'</span>) <span class="keyword">as</span> f:</div><div class="line"> <span class="keyword">for</span> line <span class="keyword">in</span> f: <span class="comment"># 读取json文件中的行(也就是json的object) </span></div><div class="line"> datalist.append(json.loads(line)) <span class="comment"># 将json的object转成 Python的dict,追加到Python 列表中, 结果都是unicode格式:[{},{},{},{},{}] </span></div><div class="line"><span class="keyword">for</span> dict <span class="keyword">in</span> datalist:</div><div class="line"> <span class="keyword">print</span> dict <span class="comment"># 打印显示 转换后的结果</span></div><div class="line"></div><div class="line"><span class="keyword">for</span> dict <span class="keyword">in</span> datalist:</div><div class="line"> dict[<span class="string">u'LOCAL'</span>] = dict[<span class="string">u'LOCAL'</span>].replace(<span class="string">'\r\n'</span>,<span class="string">'\\r\\n'</span>).replace(<span class="string">"'s"</span>,<span class="string">"\\'s"</span>) <span class="comment"># 将字段中的特殊:回车换行以及's 转换,方便形成sql语句</span></div><div class="line"> sql = <span class="string">"insert into db1.s1 (mobile,NAME,LOCAL,CreateTime,id) values('%s','%s','%s','%s','%s');"</span> % (dict[<span class="string">u'mobile'</span>],dict[<span class="string">u'NAME'</span>],dict[<span class="string">u'LOCAL'</span>],str(dict[<span class="string">u'CreateTime'</span>]),dict[<span class="string">u'id'</span>])</div><div class="line"> <span class="keyword">print</span> sql</div></pre></td></tr></table></figure>
<p>本文转自:<a href="http://www.cnblogs.com/cuisi/p/6296224.html" target="_blank" rel="external">cnblogs</a></p>
]]></content>
<summary type="html">
<blockquote>
<p>今天碰到了一个比较奇葩的需求,将mysql表数据转换成json格式。我顺便研究了一下,写了个python。</p>
</blockquote>
<hr>
</summary>
<category term="databases" scheme="http://feiyang.li/tags/databases/"/>
<category term="python" scheme="http://feiyang.li/tags/python/"/>
</entry>
<entry>
<title>从零开始搭建Hadoop2.7.1的分布式集群</title>
<link href="http://feiyang.li/2017/05/28/hadoop/index.html"/>
<id>http://feiyang.li/2017/05/28/hadoop/index.html</id>
<published>2017-05-28T11:57:31.000Z</published>
<updated>2017-05-28T12:03:20.830Z</updated>
<content type="html"><![CDATA[<script src="/assets/js/DPlayer.min.js"> </script><script src="/assets/js/APlayer.min.js"> </script><blockquote>
<p>Hadoop2.7.1,Hadoop的环境配置不是特别的复杂,但是确实有很多细节需要注意,不然会造成许多配置错误的情况。尽量保证一次配置正确防止反复修改。</p>
</blockquote>
<hr>
<a id="more"></a>
<h1 id="操作环境说明"><a href="#操作环境说明" class="headerlink" title="操作环境说明"></a>操作环境说明</h1><h2 id="操作系统"><a href="#操作系统" class="headerlink" title="操作系统"></a>操作系统</h2><ul>
<li>操作系统: window10</li>
<li>cpu: i7-6700k</li>
<li>内存:16g<figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://cdn.mmmxcc.cn/blog/20170528/171515618.png?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="" title="">
</div>
<div class="image-caption"></div>
</figure>
</li>
</ul>
<h2 id="虚拟机版本:VMware12"><a href="#虚拟机版本:VMware12" class="headerlink" title="虚拟机版本:VMware12"></a>虚拟机版本:VMware12</h2><figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://cdn.mmmxcc.cn/blog/20170528/171637610.png?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="" title="">
</div>
<div class="image-caption"></div>
</figure>
<p>ssh-keygen -t rsa -C “mmmwhy@qq.com”</p>
<h1 id="材料准备"><a href="#材料准备" class="headerlink" title="材料准备"></a>材料准备</h1><ul>
<li>ubuntu-14.10-desktop-amd64.iso (Ubuntu 光盘映像)</li>
<li><a href="https://mirrors.tuna.tsinghua.edu.cn/apache/hadoop/common/hadoop-2.7.1/hadoop-2.7.1.tar.gz" target="_blank" rel="external">hadoop-2.7.1.tar.gz (Hadoop 环境包)</a><br>注意:<br>ubuntu14.04有自带的java,版本是1.7,输入<code>java -version</code>即可查看<figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://cdn.mmmxcc.cn/blog/20170528/172116248.png?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="mark" title="">
</div>
<div class="image-caption">mark</div>
</figure>
如果没有安装java的话,网上找个安装java的文章即可,比如<a href="http://www.cnblogs.com/fnng/archive/2013/01/30/2883815.html" target="_blank" rel="external">这篇</a></li>
</ul>
<h1 id="搭建开始"><a href="#搭建开始" class="headerlink" title="搭建开始"></a>搭建开始</h1><h1 id="搭建开始-1"><a href="#搭建开始-1" class="headerlink" title="搭建开始"></a>搭建开始</h1><h2 id="Vmware-创建虚拟机"><a href="#Vmware-创建虚拟机" class="headerlink" title="Vmware 创建虚拟机"></a>Vmware 创建虚拟机</h2><figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://cdn.mmmxcc.cn/blog/20170528/172412587.png?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="" title="">
</div>
<div class="image-caption"></div>
</figure>
<p>根据Hadoop的调度规则,我们将使用VMware 12 加载 ubuntu….iso来创建三个Ubuntu 虚拟机。创建用典型安装即可。安装位置请选一个足够大的硬盘,预估需要50g。<strong>记住这个位置</strong><br><figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://cdn.mmmxcc.cn/blog/20170528/173230345.png?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="" title="">
</div>
<div class="image-caption"></div>
</figure></p>
<p>这时我们先创建一个Ubuntu虚拟机,为什么这样稍后解释。</p>
<h1 id="SSH-安装以及配置"><a href="#SSH-安装以及配置" class="headerlink" title="SSH 安装以及配置"></a>SSH 安装以及配置</h1><h2 id="先配置ubuntu镜像源"><a href="#先配置ubuntu镜像源" class="headerlink" title="先配置ubuntu镜像源"></a>先配置ubuntu镜像源</h2><h3 id="打开软件源文件"><a href="#打开软件源文件" class="headerlink" title="打开软件源文件"></a>打开软件源文件</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">sudo gedit /etc/apt/sources.list</div></pre></td></tr></table></figure>
<h3 id="将以下内容替换到源文件(注意在清华大学网站上选择合适自己系统的版本)"><a href="#将以下内容替换到源文件(注意在清华大学网站上选择合适自己系统的版本)" class="headerlink" title="将以下内容替换到源文件(注意在清华大学网站上选择合适自己系统的版本)"></a>将以下内容替换到源文件(注意在清华大学网站上选择合适自己系统的版本)</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div></pre></td><td class="code"><pre><div class="line"># 默认注释了源码镜像以提高 apt update 速度,如有需要可自行取消注释</div><div class="line">deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ trusty main restricted universe multiverse</div><div class="line"># deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ trusty main restricted universe multiverse</div><div class="line">deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ trusty-updates main restricted universe multiverse</div><div class="line"># deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ trusty-updates main restricted universe multiverse</div><div class="line">deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ trusty-backports main restricted universe multiverse</div><div class="line"># deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ trusty-backports main restricted universe multiverse</div><div class="line">deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ trusty-security main restricted universe multiverse</div><div class="line"># deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ trusty-security main restricted universe multiverse</div><div class="line"></div><div class="line"># 预发布软件源,不建议启用</div><div class="line"># deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ trusty-proposed main restricted universe multiverse</div><div class="line"># deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ trusty-proposed main restricted universe multiverse</div></pre></td></tr></table></figure>
<h2 id="更新软件源"><a href="#更新软件源" class="headerlink" title="更新软件源"></a>更新软件源</h2><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">sudo apt-get update</div></pre></td></tr></table></figure>
<h2 id="安装ssh"><a href="#安装ssh" class="headerlink" title="安装ssh"></a>安装ssh</h2><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">sudo apt-get install openssh-server</div></pre></td></tr></table></figure>
<p>已有ssh或者安装成功了的输入命令<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">ps -e | grep ssh</div></pre></td></tr></table></figure></p>
<figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://cdn.mmmxcc.cn/blog/20170528/172819756.png?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="" title="">
</div>
<div class="image-caption"></div>
</figure>
<h2 id="验证SSH是否成功安装输入"><a href="#验证SSH是否成功安装输入" class="headerlink" title="验证SSH是否成功安装输入"></a>验证SSH是否成功安装输入</h2><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">ssh localhost</div></pre></td></tr></table></figure>
<p>出现以下提示说明安装成功<br><figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://cdn.mmmxcc.cn/blog/20170528/172945211.png?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="" title="">
</div>
<div class="image-caption"></div>
</figure><br>注意,这个时候不是特权模式,特权模式反而会报错,因为我们设置的账户名其实不是<code>root</code></p>
<h2 id="生成密钥Pair"><a href="#生成密钥Pair" class="headerlink" title="生成密钥Pair"></a>生成密钥Pair</h2><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">ssh-keygen -t rsa</div></pre></td></tr></table></figure>
<p>输入后一直回车选择默认即可<br><figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://cdn.mmmxcc.cn/blog/20170528/173055646.png?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="" title="">
</div>
<div class="image-caption"></div>
</figure><br>mater主机中输入命令复制一份公钥到home中<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">cp .ssh/id_rsa.pub ~/id_rsa_master.pub</div></pre></td></tr></table></figure></p>
<h1 id="复制出两个虚拟机"><a href="#复制出两个虚拟机" class="headerlink" title="复制出两个虚拟机"></a>复制出两个虚拟机</h1><p>到开始我们放置虚拟机的位置,复制两份放在旁边,然后用vm直接打开那两个文件夹。<br>这样可以生成三个一毛一样的虚拟机。<br><figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://cdn.mmmxcc.cn/blog/20170528/173515432.png?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="" title="">
</div>
<div class="image-caption"></div>
</figure><br>顺便给他们把名字改成各自的角色。<br>这样做的好处是不需要将之前的步骤重复进行多次,不容易出错。</p>
<h1 id="配置-Hadoop-以下配置将仅仅在master主机上进行"><a href="#配置-Hadoop-以下配置将仅仅在master主机上进行" class="headerlink" title="配置 Hadoop (以下配置将仅仅在master主机上进行 )"></a>配置 Hadoop (<strong>以下配置将仅仅在master主机上进行 </strong>)</h1><ul>
<li>将解压缩好的hadoop-2.7.1文件夹拷贝到home根目录下面</li>
<li><p>在hadoop-2.7.1文件夹下创建文件,输入</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div></pre></td><td class="code"><pre><div class="line">mkdir hadoop-2.7.1/tmp</div><div class="line">mkdir hadoop-2.7.1/hdfs</div><div class="line">mkdir hadoop-2.7.1/hdfs/name</div><div class="line">mkdir hadoop-2.7.1/hdfs/data</div></pre></td></tr></table></figure>
</li>
<li><p>输入命令查看ip地址 ,三个都要看,找个小本本记下。</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">ifconfig -a</div></pre></td></tr></table></figure>
</li>
</ul>
<figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://cdn.mmmxcc.cn/blog/20170528/174451086.png?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="" title="">
</div>
<div class="image-caption"></div>
</figure>
<p>eg. 我所使用的IP地址<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div></pre></td><td class="code"><pre><div class="line">192.168.232.36 master</div><div class="line">192.168.232.54 slave1</div><div class="line">192.168.232.79 slave2</div></pre></td></tr></table></figure></p>
<ul>
<li>修改hosts<figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">sudo gedit /etc/hosts</div></pre></td></tr></table></figure>
</li>
</ul>
<figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://cdn.mmmxcc.cn/blog/20170528/174509453.png?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="" title="">
</div>
<div class="image-caption"></div>
</figure>
<p>三个ip地址和对应的角色写在hosts内</p>
<ul>
<li>修改hostname<figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">sudo gedit /etc/hostname</div></pre></td></tr></table></figure>
</li>
</ul>
<p>master 的改为 master<br>slave1 的改为 slave1<br>slave2 的改为 slave2</p>
<h2 id="修改环境变量"><a href="#修改环境变量" class="headerlink" title="修改环境变量"></a>修改环境变量</h2><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">cd ~/hadoop-2.7.1/</div></pre></td></tr></table></figure>
<h2 id="hadoop-env-sh"><a href="#hadoop-env-sh" class="headerlink" title="hadoop-env.sh"></a>hadoop-env.sh</h2><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">gedit etc/hadoop/hadoop-env.sh</div></pre></td></tr></table></figure>
<p>找到JAVA_HOME=… 一行修改为Java HOME的路径<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-amd64</div></pre></td></tr></table></figure></p>
<h2 id="core-site-xml"><a href="#core-site-xml" class="headerlink" title="core-site.xml"></a>core-site.xml</h2><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">gedit etc/hadoop/core-site.xml</div></pre></td></tr></table></figure>
<p>在configuration标签中添加<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div></pre></td><td class="code"><pre><div class="line"><property></div><div class="line"> <name>fs.defaultFS</name></div><div class="line"> <value>hdfs://master:9000</value></div><div class="line"></property></div><div class="line"></div><div class="line"><property></div><div class="line"> <name>hadoop.tmp.dir</name></div><div class="line"> <value>file:/home/mmmwhy/hadoop-2.7.1/tmp</value></div><div class="line"></property></div></pre></td></tr></table></figure></p>
<h2 id="mapred-site-xml"><a href="#mapred-site-xml" class="headerlink" title="mapred-site.xml"></a>mapred-site.xml</h2><p>创建并编辑<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div></pre></td><td class="code"><pre><div class="line">cp etc/hadoop/mapred-site.xml.template etc/hadoop/mapred-site.xml</div><div class="line">gedit etc/hadoop/mapred-site.xml</div></pre></td></tr></table></figure></p>
<p>在configuration标签中添加<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div></pre></td><td class="code"><pre><div class="line"><property></div><div class="line"> <name>fs.defaultFS</name></div><div class="line"> <value>hdfs://master:9000</value></div><div class="line"></property></div><div class="line"></div><div class="line"><property></div><div class="line"> <name>hadoop.tmp.dir</name></div><div class="line"> <value>file:/home/mmmwhy/hadoop-2.7.1/tmp</value></div><div class="line"></property></div></pre></td></tr></table></figure></p>
<h2 id="hdfs-site-xml"><a href="#hdfs-site-xml" class="headerlink" title="hdfs-site.xml"></a>hdfs-site.xml</h2><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">gedit etc/hadoop/hdfs-site.xml</div></pre></td></tr></table></figure>
<p>在configuration标签中添加<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div></pre></td><td class="code"><pre><div class="line"><property></div><div class="line"> <name>dfs.namenode.name.dir</name></div><div class="line"> <value>file:/home/mmmwhy/hadoop-2.7.1/hdfs/name</value></div><div class="line"></property></div><div class="line"><property></div><div class="line"> <name>dfs.datanode.data.dir</name></div><div class="line"> <value>file:/home/mmmwhy/hadoop-2.7.1/hdfs/data</value></div><div class="line"></property></div><div class="line"><property></div><div class="line"> <name>dfs.replication</name></div><div class="line"> <value>2</value></div><div class="line"></property></div><div class="line"><property></div><div class="line"> <name>dfs.namenode.secondary.http-address</name></div><div class="line"> <value>master:9001</value></div><div class="line"></property></div></pre></td></tr></table></figure></p>
<h2 id="yarn-site-xml"><a href="#yarn-site-xml" class="headerlink" title="yarn-site.xml"></a>yarn-site.xml</h2><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">gedit etc/hadoop/yarn-site.xml</div></pre></td></tr></table></figure>
<p>在configuration标签中添加<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div></pre></td><td class="code"><pre><div class="line"><property></div><div class="line"> <name>yarn.nodemanager.aux-services</name></div><div class="line"> <value>mapreduce_shuffle</value></div><div class="line"></property></div><div class="line"><property></div><div class="line"> <name>yarn.nodemanager.auxservices.mapreduce.shuffle.class</name></div><div class="line"> <value>org.apache.hadoop.mapred.ShuffleHandler</value></div><div class="line"></property></div><div class="line"><property></div><div class="line"> <name>yarn.resourcemanager.address</name></div><div class="line"> <value>master:8032</value></div><div class="line"></property></div><div class="line"><property></div><div class="line"> <name>yarn.resourcemanager.scheduler.address</name></div><div class="line"> <value>master:8030</value></div><div class="line"></property></div><div class="line"><property></div><div class="line"> <name>yarn.resourcemanager.resource-tracker.address</name></div><div class="line"> <value>master:8031</value></div><div class="line"></property></div><div class="line"><property></div><div class="line"> <name>yarn.resourcemanager.admin.address</name></div><div class="line"> <value>master:8033</value></div><div class="line"></property></div><div class="line"><property></div><div class="line"> <name>yarn.resourcemanager.webapp.address</name></div><div class="line"> <value>master:8088</value></div><div class="line"></property></div></pre></td></tr></table></figure></p>
<h2 id="slaves文件"><a href="#slaves文件" class="headerlink" title="slaves文件"></a>slaves文件</h2><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">gedit etc/hadoop/slaves</div></pre></td></tr></table></figure>
<p>删除原有内容,根据配置修改,此处为<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div></pre></td><td class="code"><pre><div class="line">slave1</div><div class="line">slave2</div></pre></td></tr></table></figure></p>
<ul>
<li>分发配置好的hadoop文件夹到slave1, slave2<br>前提是设置好ssh<figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div></pre></td><td class="code"><pre><div class="line">scp -r hadoop-2.7.1 mmmwhy@slave1:~/ </div><div class="line">scp -r hadoop-2.7.1 mmmwhy@slave2:~/</div></pre></td></tr></table></figure>
</li>
</ul>
<p>应该出现一大堆看起来非常厉害的东西。</p>
<ul>
<li><p>格式化hdfs<br>进入hadoop home目录</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">bin/hdfs namenode -format</div></pre></td></tr></table></figure>
</li>
<li><p>启动集群</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">sbin/start-all.sh</div></pre></td></tr></table></figure>
</li>
</ul>
<p>启动后分别在master, slave下输入jps查看进程<br><figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://cdn.mmmxcc.cn/blog/20170528/175313622.png?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="" title="">
</div>
<div class="image-caption"></div>
</figure><br>如上则说明启动成功</p>
<h1 id="集群测试"><a href="#集群测试" class="headerlink" title="集群测试"></a>集群测试</h1><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div></pre></td><td class="code"><pre><div class="line">echo hellecho hello world >> in1.txt hadoop >> in2.txt</div><div class="line">cd ~/mmmwhy/hadoop-2.7.1/</div><div class="line">bin/hadoop fs -mkdir /usr</div><div class="line">bin/hadoop fs -mkdir /usr/input</div><div class="line">bin/hadoop fs -put in1.txt /usr/input</div><div class="line">bin/hadoop fs -put in2.txt /usr/input</div><div class="line">#执行Hadoop内置example测试程序</div><div class="line">bin/hadoop jar ~/hadoop-2.7.1/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.1.jar wordcount /usr/input /usr/output</div></pre></td></tr></table></figure>
<p>然后进入 <a href="http://localhost:50070/explorer.html#/usr/input" target="_blank" rel="external">http://localhost:50070/explorer.html#/usr/input</a><br>就可以看到<br><figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://cdn.mmmxcc.cn/blog/20170528/194607098.png?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="" title="">
</div>
<div class="image-caption"></div>
</figure></p>
<h1 id="运行wordcount测试"><a href="#运行wordcount测试" class="headerlink" title="运行wordcount测试"></a>运行wordcount测试</h1><p>进入hadoop home目录<br>在hdfs(分布式文件系统)中创建一个名为input的文件夹<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">bin/hadoop fs -mkdir /input</div></pre></td></tr></table></figure></p>
<p>查看文件是否被创建<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">bin/hadoop fs -ls /</div></pre></td></tr></table></figure></p>
<p>hadoop home 下创建一个inputfile 文件夹,并在inputfile里创建两个文件<br><strong>in1.txt</strong><br><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">Hello world hello hadoop</div></pre></td></tr></table></figure></p>
<p><strong>in2.txt</strong><br><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">Hello Hadoop hello whatever</div></pre></td></tr></table></figure></p>
<p>上传两个文件进input<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">bin/hadoop fs -put inputfile/*.txt /input</div></pre></td></tr></table></figure></p>
<p>查看输入文件是否传入<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">bin/hadoop fs -ls /input</div></pre></td></tr></table></figure></p>
<figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://cdn.mmmxcc.cn/blog/20170528/195114806.png?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="" title="">
</div>
<div class="image-caption"></div>
</figure>
<p>用hadoop jar命令运行Hadoop自带的wordcount<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.1.jar wordcount /input /output</div></pre></td></tr></table></figure></p>
<figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://cdn.mmmxcc.cn/blog/20170528/195202978.png?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="" title="">
</div>
<div class="image-caption"></div>
</figure>
<p>程序开始运行,成功后查看输出文件夹<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div></pre></td><td class="code"><pre><div class="line">bin/hadoop fs -ls /output</div><div class="line">bin/hadoop fs -cat /output/part-r-00000</div></pre></td></tr></table></figure></p>
<figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://cdn.mmmxcc.cn/blog/20170528/195431835.png?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="" title="">
</div>
<div class="image-caption"></div>
</figure>
<p>至此hadoop分布式集群配置完成!本文主体内容转自<a href="http://blog.csdn.net/zl007700/article/details/50533675" target="_blank" rel="external">CSDN</a>,原作者写的非常好,但是有一些小问题,做到一半的时候花式报错,因此自己写了一份,修正了其中一些问题。</p>
]]></content>
<summary type="html">
<blockquote>
<p>Hadoop2.7.1,Hadoop的环境配置不是特别的复杂,但是确实有很多细节需要注意,不然会造成许多配置错误的情况。尽量保证一次配置正确防止反复修改。</p>
</blockquote>
<hr>
</summary>
<category term="教程" scheme="http://feiyang.li/tags/%E6%95%99%E7%A8%8B/"/>
<category term="linux" scheme="http://feiyang.li/tags/linux/"/>
<category term="hadoop" scheme="http://feiyang.li/tags/hadoop/"/>
</entry>
<entry>
<title>算法实验(一)</title>
<link href="http://feiyang.li/2017/05/24/algorithm1/index.html"/>
<id>http://feiyang.li/2017/05/24/algorithm1/index.html</id>
<published>2017-05-24T10:48:48.000Z</published>
<updated>2017-05-31T01:40:39.589Z</updated>
<content type="html"><![CDATA[<script src="/assets/js/DPlayer.min.js"> </script><script src="/assets/js/APlayer.min.js"> </script><blockquote>
<p>快速排序与背包问题相关算法分析</p>
</blockquote>
<hr>
<a id="more"></a>
<h1 id="快速排序"><a href="#快速排序" class="headerlink" title="快速排序"></a>快速排序</h1><p><a href="http://feiyang.li/2017/03/09/Python-quick-order/index.html">相关文章</a></p>
<h2 id="普通快速排序"><a href="#普通快速排序" class="headerlink" title="普通快速排序"></a>普通快速排序</h2><h3 id="算法效率:"><a href="#算法效率:" class="headerlink" title="算法效率:"></a>算法效率:</h3><ul>
<li>最坏情况下分析:当输入序列是正序或者是反序的时候,效率最坏,这时效率是$Θ(n^2)$</li>
<li>最优情况下分析:其他情况下的算法效率趋向于$Θ(nlgn)$<h3 id="伪代码"><a href="#伪代码" class="headerlink" title="伪代码"></a>伪代码</h3>对整个数组进行递归排序:<figure class="highlight cpp"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div></pre></td><td class="code"><pre><div class="line">QUICKSORT(A,p,r)</div><div class="line"><span class="keyword">if</span> p < r:</div><div class="line"> q = PARTITION(A,p,r);</div><div class="line"> QUICKSORT(A,p,q<span class="number">-1</span>);</div><div class="line"> QUICKSORT(A,q+<span class="number">1</span>,r);</div></pre></td></tr></table></figure>
</li>
</ul>
<p>快速排序算法的关键是PARTITION过程,它对A[p..r]进行就地重排:<br><figure class="highlight cpp"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div></pre></td><td class="code"><pre><div class="line">PARTITION(A,p,r)</div><div class="line">x = A[r]</div><div class="line">i = p<span class="number">-1</span></div><div class="line"><span class="keyword">for</span> j = p to r<span class="number">-1</span><span class="comment">// A[j]是待比较的元素</span></div><div class="line"> <span class="keyword">if</span> A[j] ≤ r<span class="comment">// 若A[j]比主元小</span></div><div class="line"> i = i + <span class="number">1</span> <span class="comment">// i往后一位(符合,不用替换;不符合i就停在这个地方,等着j过来叫唤)</span></div><div class="line"> exchange A[i] with A[j]<span class="comment">//倘若A[j]比主元小,其实是不发生交换的。</span></div><div class="line">exchange A[i+<span class="number">1</span>] with A[r]</div><div class="line"><span class="keyword">return</span> i + <span class="number">1</span></div></pre></td></tr></table></figure></p>
<h3 id="python代码"><a href="#python代码" class="headerlink" title="python代码"></a>python代码</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div></pre></td><td class="code"><pre><div class="line"><span class="comment"># coding=utf-8</span></div><div class="line"><span class="keyword">import</span> datetime</div><div class="line"><span class="keyword">import</span> sys </div><div class="line">sys.setrecursionlimit(<span class="number">1000000</span>)</div><div class="line"><span class="function"><span class="keyword">def</span> <span class="title">SET_NUMBER</span><span class="params">()</span>:</span></div><div class="line"> l = []</div><div class="line"> <span class="keyword">for</span> i <span class="keyword">in</span> range(<span class="number">0</span>,len(range(<span class="number">1</span>,<span class="number">2000</span>,<span class="number">2</span>)[::<span class="number">-1</span>])<span class="number">-1</span>):</div><div class="line"> l.append(range(<span class="number">1</span>,<span class="number">2000</span>,<span class="number">2</span>)[::<span class="number">-1</span>][i])</div><div class="line"> <span class="keyword">return</span> l</div><div class="line"> </div><div class="line"><span class="function"><span class="keyword">def</span> <span class="title">PARTITION</span><span class="params">(A,p,r)</span>:</span></div><div class="line"> x = A[r]</div><div class="line"> i = p - <span class="number">1</span></div><div class="line"> <span class="keyword">for</span> j <span class="keyword">in</span> range(p,r):</div><div class="line"> <span class="keyword">if</span> A[j]<=x:</div><div class="line"> i = i + <span class="number">1</span></div><div class="line"> A[i],A[j]=A[j],A[i]</div><div class="line"> A[i+<span class="number">1</span>],A[r] = A[r],A[i+<span class="number">1</span>]</div><div class="line"> <span class="keyword">return</span> i+<span class="number">1</span></div><div class="line"></div><div class="line"></div><div class="line"><span class="function"><span class="keyword">def</span> <span class="title">QUICKSORT</span><span class="params">(A,p,r)</span>:</span></div><div class="line"> <span class="keyword">if</span> p >= r:</div><div class="line"> <span class="keyword">return</span></div><div class="line"> q = PARTITION(A,p,r)</div><div class="line"> QUICKSORT(A,p,q<span class="number">-1</span>)</div><div class="line"> QUICKSORT(A,q+<span class="number">1</span>,r)</div><div class="line"> </div><div class="line"><span class="keyword">if</span> __name__ == <span class="string">'__main__'</span>:</div><div class="line"> l = SET_NUMBER()</div><div class="line"> time1 = datetime.datetime.now()</div><div class="line"> QUICKSORT(l,<span class="number">0</span>,len(l)<span class="number">-1</span>)</div><div class="line"> time2 = datetime.datetime.now()</div><div class="line"> print(time2-time1)</div></pre></td></tr></table></figure>
<h2 id="随机快速排序"><a href="#随机快速排序" class="headerlink" title="随机快速排序"></a>随机快速排序</h2><h3 id="算法效率"><a href="#算法效率" class="headerlink" title="算法效率"></a>算法效率</h3><p>我们已经知道,若输入本身已被排序,那么对于快排来说就糟了。那么如何避免这样的情况?一种方法时随机排列序列中的元素;另一种方法时随机地选择主元(pivot)。这便是随机化快速排序的思想,这种快排的好处是:其运行时间不依赖于输入序列的顺序。</p>
<p>经分析, 随机化快排的算法效率是$Θ(nlgn)$</p>
<h3 id="伪代码-1"><a href="#伪代码-1" class="headerlink" title="伪代码"></a>伪代码</h3><p>下面是分化(Partition)<br><figure class="highlight cpp"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div></pre></td><td class="code"><pre><div class="line">Random_Partition(<span class="built_in">vector</span><T> &A,<span class="keyword">int</span> p,<span class="keyword">int</span> q)</div><div class="line"> <span class="keyword">int</span> i=rand()%(q-p)+p; <span class="comment">//此行与快排不同、加入随机数参数器</span></div><div class="line"> Swap(A[i],A[p]); <span class="comment">//此行与快排不同、随机选取主元</span></div><div class="line"> <span class="keyword">return</span> Partition(A,p,q);<span class="comment">//此次与快速排序一样</span></div></pre></td></tr></table></figure></p>
<p>随机化快速排序<br><figure class="highlight cpp"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div></pre></td><td class="code"><pre><div class="line">Random_Quick_Sort(<span class="built_in">vector</span><T> &A,<span class="keyword">int</span> p,<span class="keyword">int</span> q)</div><div class="line"> <span class="keyword">if</span> (p<q):</div><div class="line"> <span class="keyword">int</span> i=Random_Partition(A,p,q);</div><div class="line"> Random_Quick_Sort(A,p,i<span class="number">-1</span>);</div><div class="line"> Random_Quick_Sort(A,i+<span class="number">1</span>,q);</div></pre></td></tr></table></figure></p>
<h3 id="python代码-1"><a href="#python代码-1" class="headerlink" title="python代码"></a>python代码</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div><div class="line">35</div><div class="line">36</div><div class="line">37</div><div class="line">38</div></pre></td><td class="code"><pre><div class="line"><span class="comment"># coding=utf-8</span></div><div class="line"><span class="keyword">import</span> datetime</div><div class="line"><span class="keyword">import</span> random</div><div class="line"><span class="function"><span class="keyword">def</span> <span class="title">SET_NUMBER</span><span class="params">()</span>:</span></div><div class="line"> l = []</div><div class="line"> <span class="keyword">for</span> i <span class="keyword">in</span> range(<span class="number">0</span>,len(range(<span class="number">1</span>,<span class="number">2000</span>,<span class="number">2</span>)[::<span class="number">-1</span>])<span class="number">-1</span>):</div><div class="line"> l.append(range(<span class="number">1</span>,<span class="number">2000</span>,<span class="number">2</span>)[::<span class="number">-1</span>][i])</div><div class="line"> <span class="keyword">return</span> l</div><div class="line"></div><div class="line"></div><div class="line"><span class="function"><span class="keyword">def</span> <span class="title">PARTITION</span><span class="params">(A,p,r)</span>:</span></div><div class="line"> x = A[r]</div><div class="line"> i = p - <span class="number">1</span></div><div class="line"> <span class="keyword">for</span> j <span class="keyword">in</span> range(p,r):</div><div class="line"> <span class="keyword">if</span> A[j]<=x:</div><div class="line"> i = i + <span class="number">1</span></div><div class="line"> A[i],A[j]=A[j],A[i]</div><div class="line"> A[i+<span class="number">1</span>],A[r] = A[r],A[i+<span class="number">1</span>]</div><div class="line"> <span class="keyword">return</span> i+<span class="number">1</span></div><div class="line"></div><div class="line"><span class="function"><span class="keyword">def</span> <span class="title">Random_Partition</span><span class="params">(A,p,r)</span>:</span></div><div class="line"> i = random.randint(p, r) </div><div class="line"> A[i],A[p]=A[p],A[i]</div><div class="line"> <span class="keyword">return</span> PARTITION(A,p,r)</div><div class="line"></div><div class="line"><span class="function"><span class="keyword">def</span> <span class="title">QUICKSORT</span><span class="params">(A,p,r)</span>:</span></div><div class="line"> <span class="keyword">if</span> p >= r:</div><div class="line"> <span class="keyword">return</span></div><div class="line"> q = Random_Partition(A,p,r)</div><div class="line"> QUICKSORT(A,p,q<span class="number">-1</span>)</div><div class="line"> QUICKSORT(A,q+<span class="number">1</span>,r)</div><div class="line"> </div><div class="line"><span class="keyword">if</span> __name__ == <span class="string">'__main__'</span>:</div><div class="line"> l = SET_NUMBER()</div><div class="line"> time1 = datetime.datetime.now()</div><div class="line"> QUICKSORT(l,<span class="number">0</span>,len(l)<span class="number">-1</span>)</div><div class="line"> time2 = datetime.datetime.now()</div><div class="line"> print(time2-time1)</div></pre></td></tr></table></figure>
<h2 id="时间效率"><a href="#时间效率" class="headerlink" title="时间效率"></a>时间效率</h2><ul>
<li>对于倒序数组<code>range(1,2000,2)[::-1]</code>(1000个数字)<ul>
<li>使用正常快速排序时间为:0:00:00.082566</li>
<li>使用随机快速排序时间为:0:00:00.005815</li>
</ul>
</li>
<li>对于随机数组<code>random.randint(0,9999)</code>(重复1000次)<ul>
<li>使用正常快速排序时间为:0:00:00.002974</li>
<li>使用随机快速排序时间为:0:00:00.004901</li>
</ul>
</li>
</ul>
<h2 id="分析性能差异"><a href="#分析性能差异" class="headerlink" title="分析性能差异"></a>分析性能差异</h2><p>快速排序的平均时间复杂度为$O(nlogn)$,最坏时间时间可达到$O(n^2)$,最坏情况是当要排序的数列基本有序的时候。根据快速排序的工作原理我们知道,选取第一个或者最后一个元素作为主元,快速排序依次将数列分成两个数列,然后递归将分开的数列进行排序。</p>
<p>当把数列平均分成两个等长的数列时,效率是最高的,如果相差太大,效率就会降低。 </p>
<p>我们通过使用随机化的快速排序随机的从数列中选取一个元素与第一个,或者最后一个元素交换,这样就会使得数列有序的概率降低。</p>
<p>所以随机快速排序平均速度是比快速排序要快的,但是因为随机排序需要有设置随机数+交换元素的时间,因此总时间上不如普通快速排序。</p>
<h1 id="背包问题"><a href="#背包问题" class="headerlink" title="背包问题"></a>背包问题</h1><p><a href="http://feiyang.li/2017/03/24/dynamic-programming/index.html">相关文章</a></p>
<h2 id="动态规划知识复习"><a href="#动态规划知识复习" class="headerlink" title="动态规划知识复习"></a>动态规划知识复习</h2><h3 id="基本步骤"><a href="#基本步骤" class="headerlink" title="基本步骤"></a>基本步骤</h3><ul>
<li>找出最优解的性质,并刻画其结构特征</li>
<li>递归地定义最优解的值</li>
<li>以自底而上的方式计算出最优值</li>
<li>根据计算最优值时得到的信息,构造一个最优解。<h3 id="若给定问题具有以下性质:"><a href="#若给定问题具有以下性质:" class="headerlink" title="若给定问题具有以下性质:"></a>若给定问题具有以下性质:</h3></li>
<li><strong>最优子结构</strong>:如果一个问题的最优解中包含了其子问题的最优解,就说该问题具有最优子结构。当一个问题具有最优子结构时,提示我们动态规划法可能会适用,但是此时贪心策略可能也是适用的。</li>
<li><strong>重叠子问题</strong>:指用来解原问题的递归算法可反复地解同样的子问题,而不是总在产生新的子问题。即当一个递归算法不断地调用同一个问题时,就说该问题包含重叠子问题。此时若用分治法递归求解,则每次遇到子问题都会视为新问题,会极大地降低算法的效率,而动态规划法总是充分利用重叠子问题,对于每个子问题仅计算一次,把解保存在一个在需要时就可以查看的表中,而每次查表的时间为常数。</li>
</ul>
<h2 id="0-1背包问题"><a href="#0-1背包问题" class="headerlink" title="0-1背包问题"></a>0-1背包问题</h2><h3 id="定义问题"><a href="#定义问题" class="headerlink" title="定义问题"></a>定义问题</h3><p>问题:有n个物品,第i个物品价值为$v_i$,重量为$w_i$,其中$v_i$和$w_i$均为非负数,背包的容量为W,W为非负数。现需要考虑如何选择装入背包的物品,使装入背包的物品总价值最大。</p>
<p>该问题以形式化描述如下:</p>
<ul>
<li>目标函数为 :<figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://img.my.csdn.net/uploads/201303/24/1364096225_6763.jpg?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="" title="">
</div>
<div class="image-caption"></div>
</figure></li>
<li>约束条件为:<figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://img.my.csdn.net/uploads/201303/24/1364096448_8646.jpg?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="" title="">
</div>
<div class="image-caption"></div>
</figure>
满足约束条件的任一集合(x1,x2,…,xn)是问题的一个可行解,问题的目标是要求问题的一个最优解。考虑一个实例,假设n=5,W=17, 每个物品的价值和重量如下表所示。可将物品1,2和5装入背包,背包未满,获得价值22,此时问题解为你(1,1,0,0,1)。也可以将物品4和5装入背包,背包装满,获得价值24,此时解为(0,0,0,1,1)。<figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://img.my.csdn.net/uploads/201303/24/1364096951_8384.jpg?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="" title="">
</div>
<div class="image-caption"></div>
</figure>
</li>
</ul>
<h3 id="按步骤求解问题"><a href="#按步骤求解问题" class="headerlink" title="按步骤求解问题"></a>按步骤求解问题</h3><ul>
<li>找出最优解的性质,并刻画其结构特征<br>可以将背包问题的求解过程看作是进行一系列的决策过程,即决定哪些物品应该放入背包,哪些物品不放入背包。如果一个问题的最优解包含了物品n,即$x_n=1$,那么其余$x_1,x<em>2,…,x</em>(n-1)$一定构成子问题1,2,…,n-1在容量W-wn时的最优解。如果这个最优解不包含物品n,即$x_n$=0,那么其余$x_1,x<em>2,…,x</em>(n-1)$一定构成子问题1,2,…,n-1在容量W时的最优解。</li>
<li>递归定义最优解的值<br>根据上述分析的最优解的结构递归地定义问题最优解。设c[i,w]表示背包剩余容量为w时,i个物品导致的最优解的总价值。<br>分为三种情况:<ul>
<li>i或者w等于0,没有空间也没有物品的时候;</li>
<li>第i的物品的重量已经超过当前空间时;</li>
<li>在增加第i个物品和不增加第i个物品之间做出艰难的抉择;<figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://img.my.csdn.net/uploads/201303/24/1364126887_3091.jpg?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="" title="">
</div>
<div class="image-caption"></div>
</figure></li>
</ul>
</li>
<li>以自底而上的方式计算出最优值<br>伪代码:<figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div></pre></td><td class="code"><pre><div class="line">BOTTOM-UP-CUT-ROD(p,n)</div><div class="line">let r[0..n] be a new array</div><div class="line">r[0]= 0</div><div class="line">for j = 1 to n</div><div class="line"> q= -∞</div><div class="line"> for i = 1 to j</div><div class="line"> q = max(q, p[i]+r[j-i])</div><div class="line"> r[j]= q</div><div class="line">return r[n]</div></pre></td></tr></table></figure>
</li>
</ul>
<h3 id="python代码-2"><a href="#python代码-2" class="headerlink" title="python代码"></a>python代码</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div></pre></td><td class="code"><pre><div class="line"><span class="comment">#coding:utf-8 </span></div><div class="line"><span class="function"><span class="keyword">def</span> <span class="title">bag</span><span class="params">(n,m,w,v)</span>:</span> </div><div class="line"> res = [[<span class="number">0</span> <span class="keyword">for</span> j <span class="keyword">in</span> range(m+<span class="number">1</span>)]<span class="keyword">for</span> i <span class="keyword">in</span> range(n+<span class="number">1</span>)] <span class="comment">#n+1 行,m+1列 值为0的矩阵 </span></div><div class="line"> <span class="keyword">for</span> i <span class="keyword">in</span> range(<span class="number">1</span>,n+<span class="number">1</span>): </div><div class="line"> <span class="keyword">for</span> j <span class="keyword">in</span> range(<span class="number">1</span>,m+<span class="number">1</span>): </div><div class="line"> res[i][j] = res[i<span class="number">-1</span>][j] <span class="comment">#0->res[0][1]->res[1][1] </span></div><div class="line"> <span class="keyword">if</span> j>=w[i<span class="number">-1</span>] <span class="keyword">and</span> res[i][j]<res[i<span class="number">-1</span>][j-w[i<span class="number">-1</span>]]+v[i<span class="number">-1</span>]:<span class="comment">#如果res[i-1][j]小于res[i-1][j-w[i-1]]+v[i-1],那么res[i][j]就等于res[i-1][j],否则就等于res[i-1][j-w[i-1]]+v[i-1] </span></div><div class="line"> res[i][j] = res[i<span class="number">-1</span>][j-w[i<span class="number">-1</span>]]+v[i<span class="number">-1</span>] </div><div class="line"> <span class="keyword">return</span> res </div><div class="line"> </div><div class="line"><span class="function"><span class="keyword">def</span> <span class="title">show</span><span class="params">(n,m,w,res)</span>:</span> </div><div class="line"> <span class="keyword">print</span> (<span class="string">u"最大值为%d"</span>%res[n][m]) </div><div class="line"> x = [<span class="keyword">False</span> <span class="keyword">for</span> i <span class="keyword">in</span> range(n)] </div><div class="line"> j = m </div><div class="line"> <span class="keyword">for</span> i <span class="keyword">in</span> range(n,<span class="number">0</span>,<span class="number">-1</span>): </div><div class="line"> <span class="keyword">if</span> res[i][j]!=res[i<span class="number">-1</span>][j]: </div><div class="line"> x[i<span class="number">-1</span>] = <span class="keyword">True</span> </div><div class="line"> j-=w[i<span class="number">-1</span>] </div><div class="line"> <span class="keyword">print</span> (<span class="string">u"选择的物品为"</span>) </div><div class="line"> <span class="keyword">for</span> i <span class="keyword">in</span> range(n): </div><div class="line"> <span class="keyword">if</span> x[i]: </div><div class="line"> print(<span class="string">u"第%d个"</span>%(i+<span class="number">1</span>)) </div><div class="line"> </div><div class="line"><span class="keyword">if</span> __name__ == <span class="string">"__main__"</span>: </div><div class="line"> <span class="comment">#n种物品,承重量为m,w物品的重量,v 物品的价值 </span></div><div class="line"> n= <span class="number">5</span> </div><div class="line"> m = <span class="number">17</span> </div><div class="line"> w = [<span class="number">3</span>,<span class="number">4</span>,<span class="number">7</span>,<span class="number">8</span>,<span class="number">9</span>] </div><div class="line"> v = [<span class="number">4</span>,<span class="number">5</span>,<span class="number">10</span>,<span class="number">11</span>,<span class="number">13</span>] </div><div class="line"> res = bag(n,m,w,v) </div><div class="line"> show(n,m,w,res)</div></pre></td></tr></table></figure>
<h2 id="部分背包问题"><a href="#部分背包问题" class="headerlink" title="部分背包问题"></a>部分背包问题</h2><p>题目要求使用贪心算法</p>
<ul>
<li>部分背包问题与0-1背包问题的差别在于:可以带走一部分。即,部分背包问题可带走的物品 是可以 无限细分的。</li>
<li>贪心策略是:总是优先选择单位重量下价值最大的物品<h3 id="具体题目"><a href="#具体题目" class="headerlink" title="具体题目"></a>具体题目</h3>假设背包可容纳50Kg的重量,物品信息如下:</li>
</ul>
<table>
<thead>
<tr>
<th>物品 i</th>
<th style="text-align:left">重量(Kg)</th>
<th>价值</th>
<th style="text-align:left">单位重量的价值</th>
</tr>
</thead>
<tbody>
<tr>
<td>1</td>
<td style="text-align:left">10</td>
<td>60</td>
<td style="text-align:left">6</td>
</tr>
<tr>
<td>2</td>
<td style="text-align:left">20</td>
<td>100</td>
<td style="text-align:left">5</td>
</tr>
<tr>
<td>3</td>
<td style="text-align:left">30</td>
<td>120</td>
<td style="text-align:left">4</td>
</tr>
</tbody>
</table>
<h3 id="伪代码-2"><a href="#伪代码-2" class="headerlink" title="伪代码"></a>伪代码</h3><p>这个看起来就很简单,先装单价最高的1,剩下的地方装2,再剩下的地方装3<br>我也不知道怎么写伪代码,直接写python吧</p>
<h3 id="C代码"><a href="#C代码" class="headerlink" title="C代码"></a>C代码</h3><figure class="highlight cpp"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div><div class="line">35</div><div class="line">36</div><div class="line">37</div><div class="line">38</div><div class="line">39</div><div class="line">40</div><div class="line">41</div><div class="line">42</div><div class="line">43</div><div class="line">44</div><div class="line">45</div><div class="line">46</div><div class="line">47</div><div class="line">48</div><div class="line">49</div><div class="line">50</div><div class="line">51</div><div class="line">52</div><div class="line">53</div><div class="line">54</div><div class="line">55</div><div class="line">56</div></pre></td><td class="code"><pre><div class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><stdio.h></span> </span></div><div class="line"><span class="meta">#<span class="meta-keyword">include</span> <span class="meta-string"><algorithm></span> </span></div><div class="line"><span class="keyword">using</span> <span class="keyword">namespace</span> <span class="built_in">std</span>; </div><div class="line"> </div><div class="line"> </div><div class="line"><span class="comment">//物品结构体,包含两个属性,w表示重量,v表示价值 </span></div><div class="line"><span class="keyword">struct</span> Good{ </div><div class="line"> <span class="keyword">int</span> id; </div><div class="line"> <span class="keyword">double</span> w; </div><div class="line"> <span class="keyword">double</span> v; </div><div class="line">}; </div><div class="line"> </div><div class="line"><span class="comment">//排序比较函数,以物品的价值/重量比值降序排序 </span></div><div class="line"><span class="function"><span class="keyword">bool</span> <span class="title">cmp</span><span class="params">(Good a, Good b)</span></span>{ </div><div class="line"> <span class="keyword">if</span> (a.v/a.w > b.v/b.w) { </div><div class="line"> <span class="keyword">return</span> <span class="literal">true</span>; </div><div class="line"> } </div><div class="line"> <span class="keyword">return</span> <span class="literal">false</span>; </div><div class="line">} </div><div class="line"> </div><div class="line"><span class="comment">//结构体数组,所有物品信息 </span></div><div class="line"><span class="keyword">struct</span> Good goods[] = {{<span class="number">1</span>,<span class="number">10</span>,<span class="number">12</span>},{<span class="number">2</span>,<span class="number">9</span>,<span class="number">9</span>},{<span class="number">3</span>,<span class="number">11</span>,<span class="number">7</span>},{<span class="number">4</span>,<span class="number">12</span>,<span class="number">9</span>},{<span class="number">5</span>,<span class="number">6</span>,<span class="number">8</span>},{<span class="number">6</span>,<span class="number">3</span>,<span class="number">4</span>}}; </div><div class="line"> </div><div class="line"> </div><div class="line"><span class="comment">//背包总重量 </span></div><div class="line"><span class="keyword">double</span> totalW = <span class="number">30</span>; </div><div class="line"> </div><div class="line"><span class="function"><span class="keyword">int</span> <span class="title">main</span><span class="params">()</span> </span></div><div class="line">{ </div><div class="line"> <span class="comment">//以物品的价值/重量比值降序排序 </span></div><div class="line"> sort(goods,goods+<span class="number">6</span>,cmp); </div><div class="line"> <span class="keyword">double</span> leftW = totalW; </div><div class="line"> <span class="keyword">int</span> totalV = <span class="number">0</span>; </div><div class="line"> </div><div class="line"> <span class="comment">//遍历排好序的物品数组 </span></div><div class="line"> <span class="keyword">for</span> (<span class="keyword">int</span> i=<span class="number">0</span>; i<<span class="number">6</span>; i++) { </div><div class="line"> <span class="comment">//如果当前背包所能承受的重量大于i物品的总量 </span></div><div class="line"> <span class="comment">//那么把i物品全部放进去 </span></div><div class="line"> <span class="keyword">if</span> (leftW >= goods[i].w) { </div><div class="line"> leftW -= goods[i].w; </div><div class="line"> totalV += goods[i].v; </div><div class="line"> <span class="built_in">printf</span>(<span class="string">"choose good[id = %d], %.1f weight,make %.1f value\n"</span>,goods[i].id,goods[i].w,goods[i].v); </div><div class="line"> </div><div class="line"> <span class="comment">//如果不能,那么取当前背包所能承受重量的相应数量物品 </span></div><div class="line"> <span class="comment">//当然价值也得按照比例来 </span></div><div class="line"> }<span class="keyword">else</span> { </div><div class="line"> totalV += leftW/goods[i].w * goods[i].v; </div><div class="line"> leftW = <span class="number">0</span>; </div><div class="line"> <span class="built_in">printf</span>(<span class="string">"choose good[id = %d], %.1f weight,make %.1f value\n"</span>,goods[i].id,leftW,leftW/goods[i].w * goods[i].v); </div><div class="line"> <span class="keyword">break</span>; </div><div class="line"> } </div><div class="line"> } </div><div class="line"> </div><div class="line"> <span class="built_in">printf</span>(<span class="string">"max total value:%d\n"</span>,totalV); </div><div class="line"> <span class="keyword">return</span> <span class="number">0</span>; </div><div class="line">}</div></pre></td></tr></table></figure>]]></content>
<summary type="html">
<blockquote>
<p>快速排序与背包问题相关算法分析</p>
</blockquote>
<hr>
</summary>
<category term="Python" scheme="http://feiyang.li/tags/Python/"/>
<category term="Algorithms" scheme="http://feiyang.li/tags/Algorithms/"/>
</entry>
<entry>
<title>Coursera ML(13)-SVM(Support Vector Machines)</title>
<link href="http://feiyang.li/2017/05/22/Coursera-ML-13/index.html"/>
<id>http://feiyang.li/2017/05/22/Coursera-ML-13/index.html</id>
<published>2017-05-22T12:25:02.000Z</published>
<updated>2017-05-22T12:32:31.116Z</updated>
<content type="html"><![CDATA[<script src="/assets/js/DPlayer.min.js"> </script><script src="/assets/js/APlayer.min.js"> </script><blockquote>
<p>本节笔记对应<a href="https://www.coursera.org/learn/machine-learning/home/week/7" target="_blank" rel="external">第七周课程</a>和西瓜书<strong>第六章 支持向量机</strong>,主要讲解了支持向量机。</p>
</blockquote>
<hr>
<a id="more"></a>
<h1 id="SVM介绍"><a href="#SVM介绍" class="headerlink" title="SVM介绍"></a>SVM介绍</h1><p>这一部分Coursera上的内容我<strong>并没有听懂</strong>,一头雾水。</p>
<h2 id="数学上的定义"><a href="#数学上的定义" class="headerlink" title="数学上的定义"></a>数学上的定义</h2><h3 id="逻辑回归模型"><a href="#逻辑回归模型" class="headerlink" title="逻辑回归模型"></a>逻辑回归模型</h3><p>从最简单的逻辑回国模型说起<br> $$\begin{align*}& h_\theta (x) = g ( \theta^T x ) \newline \newline& z = \theta^T x \newline& g(z) = \dfrac{1}{1 + e^{-z}}\end{align*}$$<br>简单点,也就是<br>$$h_\theta (x) = \dfrac{1}{1 + e^{-\theta^T x}} $$<br>其图像为:<br><figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://cdn.mmmxcc.cn/blog/20170522/142916098.png?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="" title="">
</div>
<div class="image-caption"></div>
</figure><br>我们期望的的是底下那两行,而实际给我们的这个sigmoid函数,好像并没有那么完美,这是我们进行少许的处理。</p>
<h3 id="将模型进行抽象"><a href="#将模型进行抽象" class="headerlink" title="将模型进行抽象"></a>将模型进行抽象</h3><p>原本的 $h_\theta (x) = log \dfrac{1}{1 + e^{-\theta^T x}}$ 或者 $h_\theta (x) = log(1-\dfrac{1}{1 + e^{-\theta^T x}})$ 的图像应该是底下的黑线。<br><figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://cdn.mmmxcc.cn/blog/20170522/143132931.png?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="" title="">
</div>
<div class="image-caption"></div>
</figure></p>
<p>这里,我们定义一个新的新的符号<strong>cost</strong>,就是上边那个紫色的两条直线组成的东西。<br><strong>为什幺这么做</strong>,食屎啦,我怎么知道为什么这么做,先留着这个问题。</p>
<p>改造一番之后,我们得到了<br><figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://cdn.mmmxcc.cn/blog/20170522/145216127.png?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="" title="">
</div>
<div class="image-caption"></div>
</figure></p>
<p>这个就是支持向量机的数学定义,去他妈的,什么鬼东西,干什么了嘛,SVM就被定义出来了。<br><figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://cdn.mmmxcc.cn/blog/20170522/121959778.png?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="" title="">
</div>
<div class="image-caption"></div>
</figure></p>
<h2 id="直观化的定义"><a href="#直观化的定义" class="headerlink" title="直观化的定义"></a>直观化的定义</h2><p>给定这样一个样本集$D={X_1,y_1},{X_2,y_2},{X_3,y_3}……{X_m,y_m},y_i \in(-1,+1)$<br><figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://cdn.mmmxcc.cn/blog/20170522/150009680.png?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="" title="">
</div>
<div class="image-caption"></div>
</figure><br>我们可以找到很多条直线分割这两个区域<br><figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://cdn.mmmxcc.cn/blog/20170522/151700413.png?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="mark" title="">
</div>
<div class="image-caption">mark</div>
</figure><br>那么我们应该努力去找到哪一个呢?直观上看,那个正中间的黑线看起来很不错。</p>
<p><strong>因为该划分超平面对训练样本的局部扰动容忍度最好。</strong></p>
<ul>
<li>划分超平面可以定义为如下线性方程组:<br>$$w^Tx+b=0$$<br>其中$w=(w_1;w_2…;w_d;)$为法向量,决定平面的方向,$b$是位移量。<br>将其标记为:$(w,b)$,样本中任一点到超平面$(w,b)$的距离可以表示为:<br>$$r= \dfrac{|w^T+b|}{||w||}$$</li>
<li>假定成功分类,那: $$ \left\{
\begin{aligned}
w^T+b \geq1, y_i=+1\\
w^T+b \leq-1, y_i=-1
\end{aligned}
\right.
$$ </li>
<li><p>两个异类支持向量到超平面的距离之和(间隔)为:<br>$$\gamma= \dfrac 2{||w||}$$<br>那么如果我们可以找到最大的间隔,是不是就可以使这个分割线处于最中间的位置。</p>
</li>
<li><p>使距离最小化<br>我们换一个表达方式</p>
</li>
</ul>
<p>$$\begin{aligned}<br>\min \limits_{w,b} \dfrac 12||w||^2 \<br>s.t. y_i(w^Tx+b) \geq 1<br>\end{aligned}<br>$$<br>这就是支持向量机的基本型</p>
<p>#万能的拉格朗日-对偶问题</p>
<p>又有周boss的“简单”的数学推导了:这一次是和拉格朗日一起。基本SVM的优化目标本身是一个凸二次规划问题,可以利用优化计算包求解,但是据说拉格朗日有更高效的解法:<br>对上一节最后式子使用拉格朗日乘子法就可以得到它的对偶问题 (dual problem),对偶问题出现在线性规划中,每一个求极小的线性规划问题都有一个求极大的线性规划问题互称对偶问题,解决了一个就对应解决了其对偶问题。具体到这里,对式子的每条约束添加非负拉格朗日乘子$\alpha_i$,拉格朗日函数是<br> $$L(\boldsymbol{w},b,\boldsymbol{\alpha})=\frac{1}{2}\|\boldsymbol{w}\|^2+\sum_{i=1}^m{\alpha_i(1-y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b))}$$<br>对其去$\boldsymbol{w}$和$b$的偏导为0可得到对偶问题:<br> $$max_\boldsymbol{\alpha}\ \sum_{i=1}^m\alpha_i-\frac{1}{2}\sum_{i=1}^m\sum_{j=1}^m\alpha_i\alpha_jy_iy_j\boldsymbol{x}_i^T\boldsymbol{x}_j$$$$s.t.\ \sum_{i=1}^m{\alpha_iy_i}=0,\ \alpha_i\geq0,\ i=1,2,…,m$$<br>这又是一个二次规划问题,但通用解法规模正比于样本数,在大规模数据上开销很大,可以考虑其它高效算法如SMO (Sequential Minimal Optimization)。SMO的简单描述是其不断重复以下两个步骤直至收敛:</p>
<p>选取一对需更新的变量$\alpha_i$和$\alpha_j$<br>固定$\alpha_i$和$\alpha_j$以外的参数,求解对偶问题更新后的$\alpha_i$和$\alpha_j$。</p>
<p>SMO算法在一时间也不能得到完整的理解,希望能作为学习二次规划问题的一个复杂例子在之后研究。</p>
<h1 id="核函数-核函数"><a href="#核函数-核函数" class="headerlink" title="核函数-核函数"></a>核函数-核函数</h1><h2 id="为什么要使用核函数"><a href="#为什么要使用核函数" class="headerlink" title="为什么要使用核函数"></a>为什么要使用核函数</h2><p>原始样本空间也许并不存在一个可以正确划分两类样本的超平面,一个比较好的解决办法,试讲当前原始空间映射到一个高维空间去。根据<strong>某些大佬的研究</strong>,如果原始样本空间是有限的,那么总可以找到一个高维特性空间使样本可分。</p>
<p>其实这个问题在西瓜书上说的非常清楚,但是笔记却无法做,主要是相关的图我找不出来,好气哦。</p>
<h2 id="详细介绍"><a href="#详细介绍" class="headerlink" title="详细介绍"></a>详细介绍</h2><p>取映射函数$\phi$,特征空间划分超平面模型表示为:<br>$$f(x)=\boldsymbol{w}^T\phi(x)+b$$<br>类似的有问题变为了:<br> $$min_{\boldsymbol{w},b} \frac{1}{2}\|\boldsymbol{w}\|^2$$$$s.t.\ y_i(\boldsymbol{w}^T\phi(\boldsymbol{x}_i)+b)\geq1,\ i=1,2,…,m$$<br>其对偶问题是:<br> $$max_\boldsymbol{\alpha}\ \sum_{i=1}^m\alpha_i-\frac{1}{2}\sum_{i=1}^m\sum_{j=1}^m\alpha_i\alpha_jy_iy_j\phi(\boldsymbol{x}_i^T)\phi(\boldsymbol{x}_j)$$$$s.t.\ \sum_{i=1}^m{\alpha_iy_i}=0,\ \alpha_i\geq0,\ i=1,2,…,m$$ </p>
<p>这中间涉及了$\phi(\boldsymbol{x}_i^T)\phi(\boldsymbol{x}_j)$,是样本映射到高维空间后的内积,由于维数可能很高甚至无穷,其计算很困难。此时我们就需要找一个函数,使得:<br>$$\kappa(\boldsymbol{x_i},\boldsymbol{x_j})=\phi(\boldsymbol{x}_i^T)\phi(\boldsymbol{x}_j)$$<br>称作核函数 (kernal function)。<br>有定理表明:只要一个对称函数对应的核矩阵半正定,它就能作为核函数使用。几种常用的核函数有:</p>
<table>
<thead>
<tr>
<th>名称</th>
<th style="text-align:center">表达式</th>
<th style="text-align:left">参数</th>
</tr>
</thead>
<tbody>
<tr>
<td>线性核</td>
<td style="text-align:center">$\kappa(\boldsymbol{x}_i,\boldsymbol{x}_j)=\boldsymbol{x}_i^T\boldsymbol{x}_j$</td>
<td style="text-align:left"></td>
</tr>
<tr>
<td>多项式核</td>
<td style="text-align:center">$\kappa(\boldsymbol{x}_i,\boldsymbol{x}_j)=(\boldsymbol{x}_i^T\boldsymbol{x}_j)^d$</td>
<td style="text-align:left">$d \geq 1$为多项式的次数</td>
</tr>
<tr>
<td>高斯核</td>
<td style="text-align:center">$\kappa(\boldsymbol{x}_i,\boldsymbol{x}_j)=\exp(-\frac{∥\boldsymbol{x}_i-\boldsymbol{x}_j∥^2}{2\sigma^2})$</td>
<td style="text-align:left">$\sigma \geq 0$为高斯核的带宽</td>
</tr>
<tr>
<td>拉普拉斯核</td>
<td style="text-align:center">$\kappa(\boldsymbol{x}_i,\boldsymbol{x}_j)=\exp(-\frac{∥\boldsymbol{x}_i-\boldsymbol{x}_j∥}{\sigma})$</td>
<td style="text-align:left">$\sigma \geq 0$</td>
</tr>
<tr>
<td>Sigmoid核</td>
<td style="text-align:center">$\kappa(\boldsymbol{x}_i,\boldsymbol{x}_j)=\tanh(\beta\boldsymbol{x}_i^T\boldsymbol{x}_j+\theta)$</td>
<td style="text-align:left">$\tanh$为双曲正切函数,$\beta>0,\theta<0$</td>
</tr>
</tbody>
</table>
<p>多个核函数的线性组合,两个核函数的直积均为核函数,以及若$\kappa_1$为核函数,对于任意函数$g(\boldsymbol{x})$有:$\kappa(\boldsymbol{x},\boldsymbol{z})=g(\boldsymbol{x})\kappa_1(\boldsymbol{x},\boldsymbol{z})g(\boldsymbol{z})$也是核函数<br>如何选择核函数将样本映射到真正显示其分布规律的高维空间非常重要。</p>
<h1 id="软间隔与正则化"><a href="#软间隔与正则化" class="headerlink" title="软间隔与正则化"></a>软间隔与正则化</h1><p>实际上很多问题并不能简单地归结为线性可分线性不可分,一些问题的样本空间看来是线性可分的,但总有几个样本跑到了敌对阵营,就为此强行升高维度显然得不偿失。于是引入了软间隔 (soft margin):允许有若干个样本被线性空间划分错误。此时的优化目标变为:<br> $$min_{\boldsymbol{w},b}\ \frac{1}{2}\|\boldsymbol{w}\|^2+C\sum_{i=1}^m\ell_{0/1}(y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b)-1)$$<br>其中$C$是一个常数,可以理解问允许多少个样本被分错,而$\ell_{0/1}$是0/1损失函数,当样本被分错时,它的取值为1,否则为0。这个函数直观,简单,然而非凸,非连续,常用一些替代函数:</p>
<p>hinge损失:$\ell<em>{hinge}(z)=\max(0,1,1-z)$<br>指数损失 (exponential loss):$\ell</em>{exp}(z)=\exp(-z)$<br>对率损失 (logistic loss):$\ell_{log}(z)=\log(1+\exp(-z))$</p>
<figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://cdn.mmmxcc.cn/blog/20170522/200058441.png?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="三种常见的替代损失函数对应图像为" title="">
</div>
<div class="image-caption">三种常见的替代损失函数对应图像为</div>
</figure>
<ul>
<li>红色为:hinge损失</li>
<li>蓝色为:指数损失</li>
<li>黑色为:对率损失<br>这个地方跟最开始的时候提到的图像好像,只不过Andrew直接就把结果说出来了,并没有解释为什么罢了。</li>
</ul>
<p>代入优化目标后同样可以找到它们的对偶问题进行求解。实际上此时的优化目标可以看作两部分:第一项描述划分超平面的间隔大小,第二项描述数据集上的误差。对于误差还有更一般的形式: $$min_{f}\ \Omega(f)+C\sum_{i=1}^m\ell(f(\boldsymbol{x}_i),y_i)$$ 其中$\Omega(f)$称为结构风险,后一项称为经验风险,$C$用于对二者进行折中。结构风险方便引入领域知识和用户意图:用户希望得到何种性质的模型,同时它也有助于削减假设空间。这个式子称为正则化 (regularization)。</p>
<h1 id="回归问题-支持向量回归"><a href="#回归问题-支持向量回归" class="headerlink" title="回归问题-支持向量回归"></a>回归问题-支持向量回归</h1><p>普通的回归问题计算损失为函数预测值与真实值的差,而使用支持向量回归 (Support Vector Regression, SVR),我们容忍$\epsilon$的误差,那就相当于在预测函数两端建立了一个宽为$2\epsilon$的隔离带,在此间隔带种的样本被默认为预测正确,而在此间隔带外的样本计算它的真实值与预测函数得到值的差作为损失,就此SVR问题可形式化为:<br> $$min_{\boldsymbol{w},b}\ \frac{1}{2}\|\boldsymbol{w}\|^2+C\sum_{i=1}^m\ell_{\epsilon}(f(\boldsymbol{x}_i)-y_i)$$ </p>
<p>其中$C>0$是一个常数,$l_{0/1}$是”0/1”损失函数<br> $$\begin{equation}l_{0/1}=\left\{\begin{array}{ll}
1,\,\text{if}\quad z<0\\ 0,otherwise="" \end{array}\right.\end{equation}$$="" <="" p="">
<p>常见的损失函数有hinge损失,指数损失,对率损失(见P130).</p>
<p>采用hinge损失,引入”松弛变量”(slack variables),上式可重写为<br> $$\begin{equation}\begin{split}
\min_{\mathbf w,b,\xi}&\frac{1}{2}\|\mathbf w\|^2+C\sum_{i=1}^m\xi_i\\
\text{s.t.}&\,y_i(\mathbf{w^Tx_i}+b)\ge1-\xi_i\\
&\xi_i\ge0,\,i=1,2,\cdots,m.
\end{split}\end{equation}$$ </p>
<p>对偶问题推导以及解稀疏性证明见P132.</p>
<p>支持向量引申到回归问题上即可得到支持向量回归(Support Vector Regression, SVR).SVR假设我们能容忍$f(\mathbf x)$与$y$之间最多有$\epsilon$的偏差,即仅当$f(\mathbf x)$与$y$之间的差别绝对值大于$\epsilon$时才计算损失.SVR问题可化为<br> $$\begin{equation}\begin{split}
\min_{\mathbf w,b}\frac{1}{2}\|\mathbf w\|^2+C\sum_{i=1}^ml_\epsilon(f(\mathbf x_i)-y_i)
\end{split}\end{equation}$$ </p>
<p>其中 $ C $为正则化常数,$l_\epsilon$为$\epsilon$-不敏感损失函数</p>
$$\begin{equation}l_\epsilon=\left\{\begin{array}{ll}
0,\quad\text{if}|z|\le\epsilon\\
|z|-\epsilon,otherwise
\end{array}\right.\end{equation}$$
<p>对偶问题推导以及KKT条件证明见P135-137.</p>
<h1 id="核-nuclear-方法"><a href="#核-nuclear-方法" class="headerlink" title="核(nuclear)方法"></a>核(nuclear)方法</h1><p>给定训练样本,不考虑偏移项$b$,则无论SVM还是SVR,学得的模型总能表示成核函数$\kappa(\mathbf{x_i,x_j})$的线性组合.因此有更一般的”表示定理”(representer theorem).P137。</p>
<p>由此人们发展了一系列基于核函数的学习方法,统称为核方法 (kernel methods)。最常见的就是引入核函数来将线性学习器拓展为非线性学习器,从而得到核线性判别分析 (Kernelized Linear Discriminant Analysis)。</p>
</0\\></p>]]></content>
<summary type="html">
<blockquote>
<p>本节笔记对应<a href="https://www.coursera.org/learn/machine-learning/home/week/7">第七周课程</a>和西瓜书<strong>第六章 支持向量机</strong>,主要讲解了支持向量机。</p>
</blockquote>
<hr>
</summary>
<category term="Coursera ML" scheme="http://feiyang.li/categories/Coursera-ML/"/>
<category term="Machine Learning" scheme="http://feiyang.li/tags/Machine-Learning/"/>
<category term="读书笔记" scheme="http://feiyang.li/tags/%E8%AF%BB%E4%B9%A6%E7%AC%94%E8%AE%B0/"/>
</entry>
<entry>
<title>Django 笔记之三 Django 模板</title>
<link href="http://feiyang.li/2017/05/21/django3/index.html"/>
<id>http://feiyang.li/2017/05/21/django3/index.html</id>
<published>2017-05-21T08:31:52.000Z</published>
<updated>2017-05-21T08:43:14.437Z</updated>
<content type="html"><![CDATA[<script src="/assets/js/DPlayer.min.js"> </script><script src="/assets/js/APlayer.min.js"> </script><blockquote>
<p>在前面的几节中我们都是用简单的 django.http.HttpResponse 来把内容显示到网页上,本节将讲解如何使用渲染模板的方法来显示内容。</p>
</blockquote>
<hr>
<a id="more"></a>
<h1 id="模板"><a href="#模板" class="headerlink" title="模板"></a>模板</h1><p>网站模板的设计,一般的,我们做网站有一些通用的部分,比如 导航,底部,访问统计代码等等.<br>比如<strong>nav.html, bottom.html, tongji.html</strong></p>
<p>可以写一个 base.html 来包含这些通用文件(include)<br><figure class="highlight html"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div></pre></td><td class="code"><pre><div class="line"><span class="meta"><!DOCTYPE html></span></div><div class="line"><span class="tag"><<span class="name">html</span>></span></div><div class="line"><span class="tag"><<span class="name">head</span>></span></div><div class="line"> <span class="tag"><<span class="name">title</span>></span>{% block title %}默认标题{% endblock %} - 自强学堂<span class="tag"></<span class="name">title</span>></span></div><div class="line"><span class="tag"></<span class="name">head</span>></span></div><div class="line"><span class="tag"><<span class="name">body</span>></span></div><div class="line"> </div><div class="line">{% include 'nav.html' %}</div><div class="line"> </div><div class="line">{% block content %}</div><div class="line"><span class="tag"><<span class="name">div</span>></span>这里是默认内容,所有继承自这个模板的,如果不覆盖就显示这里的默认内容。<span class="tag"></<span class="name">div</span>></span></div><div class="line">{% endblock %}</div><div class="line"> </div><div class="line">{% include 'bottom.html' %}</div><div class="line"> </div><div class="line">{% include 'tongji.html' %}</div><div class="line"> </div><div class="line"><span class="tag"></<span class="name">body</span>></span></div><div class="line"><span class="tag"></<span class="name">html</span>></span></div></pre></td></tr></table></figure></p>
<p>如果需要,写足够多的 <strong>block</strong> 以便继承的模板可以重写该部分,<strong>include</strong> 是包含其它文件的内容,就是把一些网页共用的部分拿出来,重复利用,改动的时候也方便一些,还可以把广告代码放在一个单独的<strong>html</strong>中,改动也方便一些,在用到的地方<strong>include</strong>进去。其它的页面继承自<strong> base.html </strong>就好了,继承后的模板也可以在<strong> block</strong> 块中<strong> include </strong>其它的模板文件。</p>
<p>比如我们的首页 <strong>home.html</strong>,继承或者说扩展(extends)原来的 <strong>base.html</strong>,可以简单这样写,重写部分代码(默认值的那一部分不用改)<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div></pre></td><td class="code"><pre><div class="line">{% extends 'base.html' %}</div><div class="line"> </div><div class="line">{% block title %}欢迎光临首页{% endblock %}</div><div class="line"> </div><div class="line">{% block content %}</div><div class="line">{% include 'ad.html' %}</div><div class="line">这里是首页,欢迎光临</div><div class="line">{% endblock %}</div></pre></td></tr></table></figure></p>
<p>注意:模板一般放在app下的templates中,Django会自动去这个文件夹中找。但 假如我们每个app的templates中都有一个 index.html,当我们在views.py中使用的时候,直接写一个<strong> render(request, ‘index.html’)</strong>,Django 能不能找到当前 app 的 templates 文件夹中的 index.html 文件夹呢?(<strong>答案是不一定能,有可能找错</strong>)</p>
<p>##Django 模板查找机制<br> Django 查找模板的过程是在每个 app 的 templates 文件夹中找(而不只是当前 app 中的代码只在当前的 app 的 templates 文件夹中找)。各个 app 的 templates 形成一个文件夹列表,Django 遍历这个列表,一个个文件夹进行查找,当在某一个文件夹找到的时候就停止,所有的都遍历完了还找不到指定的模板的时候就是 Template Not Found (过程类似于Python找包)。这样设计有利当然也有弊,有利是的地方是一个app可以用另一个app的模板文件,弊是有可能会找错了。所以我们使用的时候在 templates 中建立一个 app 同名的文件夹,这样就好了。</p>
<p>这样,使用的时候,模板就是 “blog/index.html” 和 “mmmwhy/index.html” 这样有app作为名称的一部分,就不会混淆。</p>
<h1 id="Django-模板进阶"><a href="#Django-模板进阶" class="headerlink" title="Django 模板进阶"></a>Django 模板进阶</h1><p>本节主要讲 Django模板中的循环,条件判断,常用的标签,过滤器的使用。</p>
<h2 id="显示一个基本的字符串在网页上"><a href="#显示一个基本的字符串在网页上" class="headerlink" title="显示一个基本的字符串在网页上"></a>显示一个基本的字符串在网页上</h2><ul>
<li>views.py<figure class="highlight python"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">def</span> <span class="title">home</span><span class="params">(request)</span>:</span></div><div class="line"> string = <span class="string">u"我是一个小胖子"</span></div><div class="line"> <span class="keyword">return</span> render(request, <span class="string">'home.html'</span>, {<span class="string">'string'</span>: string})</div></pre></td></tr></table></figure>
</li>
</ul>
<p>在视图中我们传递了一个字符串名称是 string 到模板 home.html,在模板中这样使用它:</p>
<ul>
<li>home.html<figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">{{ string }}</div></pre></td></tr></table></figure>
</li>
</ul>
<figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://cdn.mmmxcc.cn/blog/20170521/113457331.png?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="" title="">
</div>
<div class="image-caption"></div>
</figure>
<h2 id="基本的-for-循环-和-List内容的显示"><a href="#基本的-for-循环-和-List内容的显示" class="headerlink" title="基本的 for 循环 和 List内容的显示"></a>基本的 for 循环 和 List内容的显示</h2><ul>
<li><p>views.py</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">def</span> <span class="title">home</span><span class="params">(request)</span>:</span></div><div class="line"> TutorialList = [<span class="string">"HTML"</span>, <span class="string">"CSS"</span>, <span class="string">"jQuery"</span>, <span class="string">"Python"</span>, <span class="string">"Django"</span>]</div><div class="line"> <span class="keyword">return</span> render(request, <span class="string">'home.html'</span>, {<span class="string">'TutorialList'</span>: TutorialList})</div></pre></td></tr></table></figure>
</li>
<li><p>home.html</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div></pre></td><td class="code"><pre><div class="line">{% <span class="keyword">for</span> i <span class="keyword">in</span> TutorialList %}</div><div class="line">{{ i }}</div><div class="line">{% endfor %} <span class="comment">#for 循环要有一个结束标记</span></div></pre></td></tr></table></figure>
</li>
</ul>
<h2 id="显示字典中内容:"><a href="#显示字典中内容:" class="headerlink" title="显示字典中内容:"></a>显示字典中内容:</h2><ul>
<li><p>views.py</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">def</span> <span class="title">home</span><span class="params">(request)</span>:</span></div><div class="line"> info_dict = {<span class="string">'site'</span>: <span class="string">u'肥羊'</span>, <span class="string">'content'</span>: <span class="string">u'小姿势'</span>}</div><div class="line"> <span class="keyword">return</span> render(request, <span class="string">'home.html'</span>, {<span class="string">'info_dict'</span>: info_dict})</div></pre></td></tr></table></figure>
</li>
<li><p>home.html</p>
<figure class="highlight html"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">站点:{{ info_dict.site }} 内容:{{ info_dict.content }}</div></pre></td></tr></table></figure>
</li>
</ul>
<p>或者可以遍历字典内容:<br><figure class="highlight html"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div></pre></td><td class="code"><pre><div class="line">{% for key, value in info_dict.items %}</div><div class="line"> {{ key }}: {{ value }}</div><div class="line">{% endfor %}</div></pre></td></tr></table></figure></p>
<h2 id="在模板进行-条件判断和-for-循环的详细操作:"><a href="#在模板进行-条件判断和-for-循环的详细操作:" class="headerlink" title="在模板进行 条件判断和 for 循环的详细操作:"></a>在模板进行 条件判断和 for 循环的详细操作:</h2><ul>
<li><p>views.py</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div></pre></td><td class="code"><pre><div class="line"><span class="function"><span class="keyword">def</span> <span class="title">home</span><span class="params">(request)</span>:</span></div><div class="line"> List = map(str, range(<span class="number">100</span>))<span class="comment"># 一个长度为100的 List</span></div><div class="line"> <span class="keyword">return</span> render(request, <span class="string">'home.html'</span>, {<span class="string">'List'</span>: List})</div></pre></td></tr></table></figure>
</li>
<li><p>home.html</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div></pre></td><td class="code"><pre><div class="line">{% <span class="keyword">for</span> item <span class="keyword">in</span> List %}</div><div class="line"> {{ item }}{% <span class="keyword">if</span> <span class="keyword">not</span> forloop.last %},{% endif %}</div><div class="line">{% endfor %}</div></pre></td></tr></table></figure>
</li>
</ul>
<figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://cdn.mmmxcc.cn/blog/20170521/134312852.png?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="" title="">
</div>
<div class="image-caption"></div>
</figure>
<h2 id="模板中-获取当前网址,当前用户等"><a href="#模板中-获取当前网址,当前用户等" class="headerlink" title="模板中 获取当前网址,当前用户等"></a>模板中 获取当前网址,当前用户等</h2><ul>
<li><p>获取当前用户</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div></pre></td><td class="code"><pre><div class="line">{% <span class="keyword">if</span> request.user.is_authenticated %}</div><div class="line"> {{ request.user.username }},您好!</div><div class="line">{% <span class="keyword">else</span> %}</div><div class="line"> 请登陆,这里放登陆链接</div><div class="line">{% endif %}</div></pre></td></tr></table></figure>
</li>
<li><p>获取当前网址</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">{{ request.path }}</div></pre></td></tr></table></figure>
</li>
<li><p>获取当前 GET 参数:</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">{{ request.GET.urlencode }}</div></pre></td></tr></table></figure></li>
</ul>
]]></content>
<summary type="html">
<blockquote>
<p>在前面的几节中我们都是用简单的 django.http.HttpResponse 来把内容显示到网页上,本节将讲解如何使用渲染模板的方法来显示内容。</p>
</blockquote>
<hr>
</summary>
<category term="Django" scheme="http://feiyang.li/categories/Django/"/>
<category term="教程" scheme="http://feiyang.li/tags/%E6%95%99%E7%A8%8B/"/>
<category term="Django" scheme="http://feiyang.li/tags/Django/"/>
</entry>
<entry>
<title>Django 笔记之二 视图与网址</title>
<link href="http://feiyang.li/2017/05/20/django2/index.html"/>
<id>http://feiyang.li/2017/05/20/django2/index.html</id>
<published>2017-05-20T08:06:53.000Z</published>
<updated>2017-05-21T08:37:15.595Z</updated>
<content type="html"><![CDATA[<script src="/assets/js/DPlayer.min.js"> </script><script src="/assets/js/APlayer.min.js"> </script><blockquote>
<p><a href="http://feiyang.li/2017/05/20/django1/index.html">上节</a>已经提到新建app,这节稍微深入一些</p>
</blockquote>
<hr>
<a id="more"></a>
<h1 id="新建app"><a href="#新建app" class="headerlink" title="新建app"></a>新建app</h1><h2 id="基本步骤"><a href="#基本步骤" class="headerlink" title="基本步骤"></a>基本步骤</h2><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">python manage.py startapp learn # learn 是一个app的名称</div></pre></td></tr></table></figure>
<p>之后大概是这个样子,也许你们会发现我用的其实是<strong>世界上最好的python IDE</strong>~<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div><div class="line">35</div><div class="line">36</div><div class="line">37</div><div class="line">38</div><div class="line">39</div><div class="line">40</div><div class="line">41</div><div class="line">42</div><div class="line">43</div><div class="line">44</div><div class="line">45</div><div class="line">46</div><div class="line">47</div><div class="line">48</div><div class="line">49</div><div class="line">50</div><div class="line">51</div><div class="line">52</div><div class="line">53</div><div class="line">54</div></pre></td><td class="code"><pre><div class="line">E:.</div><div class="line">│ db.sqlite3</div><div class="line">│ manage.py</div><div class="line">│ Python</div><div class="line">│</div><div class="line">├─.idea</div><div class="line">│ │ blog.iml</div><div class="line">│ │ misc.xml</div><div class="line">│ │ modules.xml</div><div class="line">│ │ workspace.xml</div><div class="line">│ │</div><div class="line">│ └─inspectionProfiles</div><div class="line">│ profiles_settings.xml</div><div class="line">│</div><div class="line">├─blog</div><div class="line">│ │ settings.py</div><div class="line">│ │ settings.pyc</div><div class="line">│ │ urls.py</div><div class="line">│ │ urls.pyc</div><div class="line">│ │ wsgi.py</div><div class="line">│ │ wsgi.pyc</div><div class="line">│ │ __init__.py</div><div class="line">│ │ __init__.pyc</div><div class="line">│ │</div><div class="line">│ └─__pycache__</div><div class="line">│ settings.cpython-35.pyc</div><div class="line">│ urls.cpython-35.pyc</div><div class="line">│ wsgi.cpython-35.pyc</div><div class="line">│ __init__.cpython-35.pyc</div><div class="line">│</div><div class="line">└─my_blog</div><div class="line"> │ admin.py</div><div class="line"> │ admin.pyc</div><div class="line"> │ apps.py</div><div class="line"> │ models.py</div><div class="line"> │ models.pyc</div><div class="line"> │ tests.py</div><div class="line"> │ views.py</div><div class="line"> │ views.pyc</div><div class="line"> │ __init__.py</div><div class="line"> │ __init__.pyc</div><div class="line"> │</div><div class="line"> ├─migrations</div><div class="line"> │ │ __init__.py</div><div class="line"> │ │ __init__.pyc</div><div class="line"> │ │</div><div class="line"> │ └─__pycache__</div><div class="line"> │ __init__.cpython-35.pyc</div><div class="line"> │</div><div class="line"> └─__pycache__</div><div class="line"> admin.cpython-35.pyc</div><div class="line"> models.cpython-35.pyc</div><div class="line"> views.cpython-35.pyc</div><div class="line"> __init__.cpython-35.pyc</div></pre></td></tr></table></figure></p>
<h2 id="声明my-blog"><a href="#声明my-blog" class="headerlink" title="声明my_blog"></a>声明<strong>my_blog</strong></h2><p>我们需要在<strong>settings.py</strong>中去声明一下<strong>my_blog</strong></p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div></pre></td><td class="code"><pre><div class="line">INSTALLED_APPS = [</div><div class="line"> <span class="string">'django.contrib.admin'</span>,</div><div class="line"> <span class="string">'django.contrib.auth'</span>,</div><div class="line"> <span class="string">'django.contrib.contenttypes'</span>,</div><div class="line"> <span class="string">'django.contrib.sessions'</span>,</div><div class="line"> <span class="string">'django.contrib.messages'</span>,</div><div class="line"> <span class="string">'django.contrib.staticfiles'</span>,</div><div class="line"> <span class="string">'my_blog'</span>,</div><div class="line">]</div></pre></td></tr></table></figure>
<p>新建的 app 如果不加到 INSTALL_APPS 中的话, django 就不能自动找到app中的模板文件(app-name/templates/下的文件)和静态文件(app-name/static/中的文件) 。</p>
<h2 id="定义视图内容"><a href="#定义视图内容" class="headerlink" title="定义视图内容"></a>定义视图内容</h2><p>将views.py修改为我们希望展示的内容:<br><figure class="highlight python"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div></pre></td><td class="code"><pre><div class="line"><span class="comment">#coding: utf-8</span></div><div class="line"><span class="keyword">from</span> django.http <span class="keyword">import</span> HttpResponse</div><div class="line"><span class="function"><span class="keyword">def</span> <span class="title">index</span><span class="params">(request)</span>:</span></div><div class="line"> <span class="keyword">return</span> HttpResponse(<span class="string">u'你好哇,李肥羊,希望你可以瘦下来!'</span>)</div></pre></td></tr></table></figure></p>
<p>使用<code>python manage.py runserver</code>,启动django服务</p>
<ul>
<li><p>第一行是声明编码为utf-8, 因为我们在代码中用到了中文,如果不声明就报错.</p>
</li>
<li><p>第二行引入HttpResponse,它是用来向网页返回内容的,就像Python中的 print 一样,只不过 HttpResponse 是把内容显示到网页上。</p>
</li>
</ul>
<p>我们定义了一个index()函数,第一个参数必须是 request,与网页发来的请求有关,request 变量里面包含get或post的内容,用户浏览器,系统等信息在里面(后面会讲,先了解一下就可以)。</p>
<p>函数返回了一个 HttpResponse 对象,可以经过一些处理,最终显示几个字到网页上。<br><figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://cdn.mmmxcc.cn/blog/20170520/103039383.png?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="" title="">
</div>
<div class="image-caption"></div>
</figure></p>
<p>如果你前端技术非常好的话,可以直接把整个页面return回去。。。</p>
<p>那问题来了,我们访问什么网址才能看到刚才写的这个函数呢?怎么让网址和函数关联起来呢?</p>
<h2 id="定义视图函数相关的URL-网址"><a href="#定义视图函数相关的URL-网址" class="headerlink" title="定义视图函数相关的URL(网址)"></a>定义视图函数相关的URL(网址)</h2><p>即规定 访问什么网址对应什么内容,我们打开 urls.py 这个文件, 修改其中的代码:<br><figure class="highlight python"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div></pre></td><td class="code"><pre><div class="line"><span class="keyword">from</span> django.conf.urls <span class="keyword">import</span> url</div><div class="line"><span class="keyword">from</span> django.contrib <span class="keyword">import</span> admin</div><div class="line"><span class="keyword">from</span> my_blog <span class="keyword">import</span> views <span class="keyword">as</span> my_blog_view</div><div class="line">urlpatterns = [</div><div class="line"> url(<span class="string">r'^admin/'</span>, admin.site.urls),</div><div class="line"> url(<span class="string">r'^$'</span>, my_blog_view.index),</div><div class="line">]</div></pre></td></tr></table></figure></p>
<p>先从<strong>my_blog</strong>导入<strong>views</strong>进入<strong>urls.py</strong>,然后用正则表达式,设定对应规则。如果忘了正则表达式怎么使用,也许这个<a href="http://feiyang.li/2017/02/10/Python_lambda/index.html">网页</a>会对你有帮助.</p>
<h1 id="通过链接做加减法"><a href="#通过链接做加减法" class="headerlink" title="通过链接做加减法"></a>通过链接做加减法</h1><p>跟之前的步骤一样,先修改<strong>view.py</strong>文件</p>
<h2 id="修改view-py"><a href="#修改view-py" class="headerlink" title="修改view.py"></a>修改view.py</h2><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div></pre></td><td class="code"><pre><div class="line">def add(request):</div><div class="line"> a = request.GET['a']</div><div class="line"> b = request.GET['b']</div><div class="line"> c = int(a)+int(b)</div><div class="line"> return HttpResponse(str(c))</div><div class="line"></div><div class="line">def sub(request):</div><div class="line"> a = request.GET['a']</div><div class="line"> b = request.GET['b']</div><div class="line"> c = int(a)-int(b)</div><div class="line"> return HttpResponse(str(c))</div><div class="line">def add2(request, a, b):</div><div class="line"> c = int(a) + int(b)</div><div class="line"> return HttpResponse(str(c))</div></pre></td></tr></table></figure>
<h2 id="修改urls-py"><a href="#修改urls-py" class="headerlink" title="修改urls.py"></a>修改urls.py</h2><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div></pre></td><td class="code"><pre><div class="line">url(r'^add/$', my_blog_view.add, name='add'), # 注意修改了这一行</div><div class="line">url(r'^sub/$', my_blog_view.add, name='sub'), # 注意修改了这一行</div><div class="line">url(r'^add/(\d+)/(\d+)/$', my_blog_view.add2, name='add2'),</div></pre></td></tr></table></figure>
<h2 id="运行检查"><a href="#运行检查" class="headerlink" title="运行检查"></a>运行检查</h2><p>输入<a href="http://127.0.0.1:8000/add/?a=4&b=5或者http://127.0.0.1:8000/add/?a=4&b=5" target="_blank" rel="external">http://127.0.0.1:8000/add/?a=4&b=5或者http://127.0.0.1:8000/add/?a=4&b=5</a><br><figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://cdn.mmmxcc.cn/blog/20170520/151648247.png?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="" title="">
</div>
<div class="image-caption"></div>
</figure><br>就可以获得答案</p>
<h1 id="URL-name返回整个页面"><a href="#URL-name返回整个页面" class="headerlink" title="URL name返回整个页面"></a>URL name返回整个页面</h1><h2 id="urls-py"><a href="#urls-py" class="headerlink" title="urls.py"></a>urls.py</h2><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">url(r'^add/$', calc_views.add, name='add')</div></pre></td></tr></table></figure>
<p> 这里的 name=’add’ 是用来干什么的呢,简单说,name 可以用于在 templates, models, views ……中得到对应的网址,相当于“给网址取了个名字”,只要这个名字不变,网址变了也能通过名字获取到。</p>
<h2 id="views-py"><a href="#views-py" class="headerlink" title="views.py"></a>views.py</h2><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div></pre></td><td class="code"><pre><div class="line">from django.http import HttpResponse</div><div class="line">from django.shortcuts import render</div><div class="line"> </div><div class="line"> </div><div class="line">def index(request):</div><div class="line"> return render(request, 'home.html')</div></pre></td></tr></table></figure>
<h2 id="templates"><a href="#templates" class="headerlink" title="templates"></a>templates</h2><p>我们在 my_blog 这个 app 中新建一个 templates 文件夹,在templates中新建一个 home.html<br>文件 my_blog/templates/home.html 中写入以下内容(保存时用 utf8 编码)<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div></pre></td><td class="code"><pre><div class="line"><!DOCTYPE html></div><div class="line"><html lang="en"></div><div class="line"><head></div><div class="line"> <meta charset="UTF-8"></div><div class="line"> <title>今天是个好天气</title></div><div class="line"></head></div><div class="line"><body></div><div class="line"><a href="/add/4/5/">计算 4+5</a></div><div class="line"></body></div><div class="line"></html></div></pre></td></tr></table></figure></p>
<h2 id="运行结果"><a href="#运行结果" class="headerlink" title="运行结果"></a>运行结果</h2><p>运行开发服务器,我们访问 <a href="http://127.0.0.1:8000/" target="_blank" rel="external">http://127.0.0.1:8000/</a> 可以看到<br><figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://cdn.mmmxcc.cn/blog/20170520/153435675.png?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="" title="">
</div>
<div class="image-caption"></div>
</figure><br>点击<strong>计算4+5</strong>,可以看到答案。</p>
<h2 id="如何用-Python-代码获取对应的网址"><a href="#如何用-Python-代码获取对应的网址" class="headerlink" title="如何用 Python 代码获取对应的网址"></a>如何用 Python 代码获取对应的网址</h2><p>···<br><a href="/add/4/5/">计算 4+5</a><br>···<br>如果这样写“死网址”,之后上线后,如果想要改变链接结构,将会很麻烦。如果有些地方没有修改过来,就不能使用了、<br>使用<code>python manage.py shell</code>,进入<code>django shell</code>调试页面.<br><figure class="highlight python"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div></pre></td><td class="code"><pre><div class="line"><span class="meta">>>> </span><span class="keyword">from</span> django.core.urlresolvers <span class="keyword">import</span> reverse <span class="comment"># django 1.4.x - django 1.10.x</span></div><div class="line"><span class="meta">>>> </span><span class="keyword">from</span> django.urls <span class="keyword">import</span> reverse <span class="comment"># django 1.10.x 新的,更加规范了</span></div><div class="line"> </div><div class="line"><span class="meta">>>> </span>reverse(<span class="string">'add2'</span>, args=(<span class="number">4</span>,<span class="number">5</span>))</div><div class="line"><span class="string">u'/add/4/5/'</span></div><div class="line"><span class="meta">>>> </span>reverse(<span class="string">'add2'</span>, args=(<span class="number">444</span>,<span class="number">555</span>))</div><div class="line"><span class="string">u'/add/444/555/'</span></div></pre></td></tr></table></figure></p>
<p><strong>reverse 接收 url 中的 name 作为第一个参数</strong>,我们在代码中就可以通过 reverse() 来获取对应的网址。也就是输入name返回<strong>对应网址</strong>。</p>
<p>例如:<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line"><a href="{% url 'add2' 4 5 %}"> 计算 4+5</a></div></pre></td></tr></table></figure></p>
<p>上面的代码渲染成最终的页面是</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line"><a href="/add/4/5/">计算 4+5</a></div></pre></td></tr></table></figure>
|这样就可以通过 {% url 'add2' 4 5 %} 获取到对应的网址 /add/4/5/ |<br><br>这样,即使我们在<strong>urls.py</strong>中修改了网址链接,<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">url(r'^new_add/(\d+)/(\d+)/$', my_blog_view.add2, name='add2'),</div></pre></td></tr></table></figure>
<p>注意看重点 add 变成了 new_add,但是后面的 name=’add2’ 没改,这时 |`{% url 'add2' 4 5 %}` | 就会渲染对应的网址成<code>/new_add/4/5/</code></p>
<p>用在 views.py 或 models.py 等地方的 reverse函数,同样会根据 name 对应的url获取到新的网址。</p>
<p>想要改网址的时候,修改 urls.py 中的正则表达式部分(url 参数第一部分),name 不变的前提下,其它地方都不需要修改。</p>
<h2 id="链接跳转"><a href="#链接跳转" class="headerlink" title="链接跳转"></a>链接跳转</h2><h3 id="views-py-写一个跳转的函数"><a href="#views-py-写一个跳转的函数" class="headerlink" title="views.py 写一个跳转的函数"></a>views.py 写一个跳转的函数</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div></pre></td><td class="code"><pre><div class="line">from django.http import HttpResponseRedirect</div><div class="line">from django.urls import reverse # new in django 1.10.x</div><div class="line"> </div><div class="line">def old_add2_redirect(request, a, b):</div><div class="line"> return HttpResponseRedirect(</div><div class="line"> reverse('add2', args=(a, b))</div><div class="line"> )</div></pre></td></tr></table></figure>
<h3 id="urls-py-1"><a href="#urls-py-1" class="headerlink" title="urls.py"></a>urls.py</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div></pre></td><td class="code"><pre><div class="line">url(r'^add/(\d+)/(\d+)/$', my_blog_view.old_add2_redirect),</div><div class="line">url(r'^new_add/(\d+)/(\d+)/$', my_blog_view.add2, name='add2'),</div></pre></td></tr></table></figure>
<p>即可完成跳转</p>
]]></content>
<summary type="html">
<blockquote>
<p><a href="http://feiyang.li/2017/05/20/django1/index.html">上节</a>已经提到新建app,这节稍微深入一些</p>
</blockquote>
<hr>
</summary>
<category term="Django" scheme="http://feiyang.li/categories/Django/"/>
<category term="教程" scheme="http://feiyang.li/tags/%E6%95%99%E7%A8%8B/"/>
<category term="Django" scheme="http://feiyang.li/tags/Django/"/>
</entry>
<entry>
<title>Django 笔记之一 基本命令</title>
<link href="http://feiyang.li/2017/05/20/django1/index.html"/>
<id>http://feiyang.li/2017/05/20/django1/index.html</id>
<published>2017-05-20T01:20:41.000Z</published>
<updated>2017-05-21T08:32:12.067Z</updated>
<content type="html"><![CDATA[<script src="/assets/js/DPlayer.min.js"> </script><script src="/assets/js/APlayer.min.js"> </script><p>本节主要是为了让您了解一些django最基本的命令,请尝试着记住它们,并且多多练习下,打开 Linux 或 MacOS 的 Terminal (终端)直接在 终端中输入这些命令(不是 python 的 shell中)</p>
<hr>
<a id="more"></a>
<p>如果是 windows 用 cmd(开始 搜索 cmd 或者 快捷键 win + R,输入 cmd) 直接在 cmd 上操作。</p>
<h1 id="新建一个-django-project"><a href="#新建一个-django-project" class="headerlink" title="新建一个 django project"></a>新建一个 django project</h1><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">django-admin startproject blog</div></pre></td></tr></table></figure>
<figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://upload-images.jianshu.io/upload_images/3326314-20ef9944c7e2688d.png?imageMogr2/auto-orient/strip%7CimageView2/2/w/1240?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="" title="">
</div>
<div class="image-caption"></div>
</figure>
<p>特别是在 windows 上,如果报错,尝试用 django-admin 代替 django-admin.py 试试<br>注意 project_name 是自己的项目名称,需要为合法的 Python 包名,如不能为 1a 或 a-b。</p>
<h1 id="新建-app"><a href="#新建-app" class="headerlink" title="新建 app"></a>新建 app</h1><p>要先进入项目目录下,cd project_name 然后执行下面的命令(下同,已经在项目目录下则不需要 cd project_name)<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">python manage.py startapp my_blog</div></pre></td></tr></table></figure></p>
<p>或<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">django-admin startapp my_blog</div></pre></td></tr></table></figure></p>
<p>效果太概是这个样子的<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div></pre></td><td class="code"><pre><div class="line">.</div><div class="line">├── my_blog</div><div class="line">│ ├── admin.py</div><div class="line">│ ├── __init__.py</div><div class="line">│ ├── migrations</div><div class="line">│ │ └── __init__.py</div><div class="line">│ ├── models.py</div><div class="line">│ ├── tests.py</div><div class="line">│ └── views.py</div><div class="line">├── blog</div><div class="line">│ ├── __init__.py</div><div class="line">│ ├── __pycache__</div><div class="line">│ │ ├── __init__.cpython-34.pyc</div><div class="line">│ │ └── settings.cpython-34.pyc</div><div class="line">│ ├── settings.py</div><div class="line">│ ├── urls.py</div><div class="line">│ └── wsgi.py</div><div class="line">└── manage.py</div></pre></td></tr></table></figure></p>
<p>一般一个项目有多个app, 当然通用的app也可以在多个项目中使用。</p>
<p>与项目名类似 app name 也需要为合法的 Python 包名,如 blog,news, aboutus 等都是合法的 app 名称。</p>
<h1 id="同步更改数据库表或字段"><a href="#同步更改数据库表或字段" class="headerlink" title="同步更改数据库表或字段"></a>同步更改数据库表或字段</h1><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div></pre></td><td class="code"><pre><div class="line">Django 1.7.1及以上 用以下命令</div><div class="line"># 1. 创建更改的文件</div><div class="line">python manage.py makemigrations</div><div class="line"># 2. 将生成的py文件应用到数据库</div><div class="line">python manage.py migrate</div></pre></td></tr></table></figure>
<p>这种方法可以在SQL等数据库中创建与models.py代码对应的表,不需要自己手动执行SQL。</p>
<p>备注:对已有的 models 进行修改,Django 1.7之前的版本的Django都是无法自动更改表结构的,不过有第三方工具 south,详见 Django 数据库迁移 一节。</p>
<h1 id="使用开发服务器"><a href="#使用开发服务器" class="headerlink" title="使用开发服务器"></a>使用开发服务器</h1><p>开发服务器,即开发时使用,一般修改代码后会自动重启,方便调试和开发,但是由于性能问题,建议只用来测试,不要用在生产环境。</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div></pre></td><td class="code"><pre><div class="line">python manage.py runserver</div><div class="line"> </div><div class="line"># 当提示端口被占用的时候,可以用其它端口:</div><div class="line">python manage.py runserver 8001</div><div class="line">python manage.py runserver 9999</div><div class="line">(当然也可以kill掉占用端口的进程,具体后面有讲,此处想知道的同学可查下 lsof 命令用法)</div><div class="line"> </div><div class="line"># 监听机器所有可用 ip (电脑可能有多个内网ip或多个外网ip)</div><div class="line">python manage.py runserver 0.0.0.0:8000</div><div class="line"># 如果是外网或者局域网电脑上可以用其它电脑查看开发服务器</div><div class="line"># 访问对应的 ip加端口,比如 http://172.16.20.2:8000</div></pre></td></tr></table></figure>
<h1 id="清空数据库"><a href="#清空数据库" class="headerlink" title="清空数据库"></a>清空数据库</h1><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">python manage.py flush</div></pre></td></tr></table></figure>
<p>此命令会询问是 yes 还是 no, 选择 yes 会把数据全部清空掉,只留下空表。</p>
<h1 id="创建超级管理员"><a href="#创建超级管理员" class="headerlink" title="创建超级管理员"></a>创建超级管理员</h1><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div></pre></td><td class="code"><pre><div class="line">python manage.py createsuperuser</div><div class="line"># 按照提示输入用户名和对应的密码就好了邮箱可以留空,用户名和密码必填</div><div class="line"> </div><div class="line"># 修改 用户密码可以用:</div><div class="line">python manage.py changepassword username</div></pre></td></tr></table></figure>
<h1 id="导出数据-导入数据"><a href="#导出数据-导入数据" class="headerlink" title="导出数据 导入数据"></a>导出数据 导入数据</h1><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div></pre></td><td class="code"><pre><div class="line">python manage.py dumpdata appname > appname.json</div><div class="line">python manage.py loaddata appname.json</div></pre></td></tr></table></figure>
<p>关于数据操作 详见:数据导入数据迁移,现在了解有这个用法就可以了。</p>
<h1 id="Django-项目环境终端"><a href="#Django-项目环境终端" class="headerlink" title="Django 项目环境终端"></a>Django 项目环境终端</h1><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div></pre></td><td class="code"><pre><div class="line">python manage.py shell</div><div class="line">如果你安装了 bpython 或 ipython 会自动用它们的界面,推荐安装 bpython。</div></pre></td></tr></table></figure>
<p>这个命令和 直接运行 python 或 bpython 进入 shell 的区别是:你可以在这个 shell 里面调用当前项目的 models.py 中的 API,对于操作数据,还有一些小测试非常方便。</p>
<h1 id="数据库命令行"><a href="#数据库命令行" class="headerlink" title="数据库命令行"></a>数据库命令行</h1><figure class="highlight plain"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">python manage.py dbshell</div></pre></td></tr></table></figure>
<p>Django 会自动进入在settings.py中设置的数据库,如果是 MySQL 或 postgreSQL,会要求输入数据库用户密码。</p>
<p>在这个终端可以执行数据库的SQL语句。如果您对SQL比较熟悉,可能喜欢这种方式。</p>
<h1 id="更多命令"><a href="#更多命令" class="headerlink" title="更多命令"></a>更多命令</h1><p>终端上输入 python manage.py 可以看到详细的列表,在忘记子名称的时候特别有用。</p>
]]></content>
<summary type="html">
<p>本节主要是为了让您了解一些django最基本的命令,请尝试着记住它们,并且多多练习下,打开 Linux 或 MacOS 的 Terminal (终端)直接在 终端中输入这些命令(不是 python 的 shell中)</p>
<hr>
</summary>
<category term="Django" scheme="http://feiyang.li/categories/Django/"/>
<category term="教程" scheme="http://feiyang.li/tags/%E6%95%99%E7%A8%8B/"/>
<category term="Django" scheme="http://feiyang.li/tags/Django/"/>
</entry>
<entry>
<title>Coursera ML(12)-Machine Learning System Design</title>
<link href="http://feiyang.li/2017/05/19/Coursera-ML-12-Machine-Learning-System-Design/index.html"/>
<id>http://feiyang.li/2017/05/19/Coursera-ML-12-Machine-Learning-System-Design/index.html</id>
<published>2017-05-19T10:53:03.000Z</published>
<updated>2017-05-22T06:24:40.702Z</updated>
<content type="html"><![CDATA[<script src="/assets/js/DPlayer.min.js"> </script><script src="/assets/js/APlayer.min.js"> </script><blockquote>
<p>机器学习系统设计</p>
</blockquote>
<hr>
<a id="more"></a>
<h1 id="查准率-Precision-和查全率-Recall"><a href="#查准率-Precision-和查全率-Recall" class="headerlink" title="查准率(Precision)和查全率(Recall)"></a>查准率(Precision)和查全率(Recall)</h1><figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://cdn.mmmxcc.cn/blog/20170519/112809620.png?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="" title="">
</div>
<div class="image-caption"></div>
</figure>
<p>分为四种情况:</p>
<ul>
<li>正确肯定(True Positive<strong>TP</strong>):预测为真,实际为真。</li>
<li>正确否定(True Negative <strong>TN</strong>):预测为假,实际为假。</li>
<li>错误肯定(False Positive<strong>FP</strong>):预测为真,实际为假。</li>
<li>错误否定(False Negative<strong>FN</strong>):预测为假,实际为真。<h2 id="查准率"><a href="#查准率" class="headerlink" title="查准率"></a>查准率</h2>$$Precison=\dfrac{TP}{TP+FP}$$<br>在所有我们<strong>预测有恶性肿瘤的病人</strong>中,实际上有恶性肿瘤的病人百分比,越高越好。<h2 id="查全率"><a href="#查全率" class="headerlink" title="查全率"></a>查全率</h2>$$Recall=\dfrac{TP}{TP+FN}$$<br>在所有我们<strong>实际上有病的病人</strong>中,预测成功有恶性肿瘤的病人百分比,越高越好。</li>
</ul>
<h1 id="F1值"><a href="#F1值" class="headerlink" title="F1值"></a>F1值</h1><p>$$F_1=\dfrac{PR}{P+R}$$</p>
]]></content>
<summary type="html">
<blockquote>
<p>机器学习系统设计</p>
</blockquote>
<hr>
</summary>
<category term="Coursera ML" scheme="http://feiyang.li/categories/Coursera-ML/"/>
<category term="Machine Learning" scheme="http://feiyang.li/tags/Machine-Learning/"/>
<category term="读书笔记" scheme="http://feiyang.li/tags/%E8%AF%BB%E4%B9%A6%E7%AC%94%E8%AE%B0/"/>
</entry>
<entry>
<title>coursera</title>
<link href="http://feiyang.li/2017/05/18/coursera/index.html"/>
<id>http://feiyang.li/2017/05/18/coursera/index.html</id>
<published>2017-05-18T02:14:53.000Z</published>
<updated>2017-06-02T05:04:20.210Z</updated>
<content type="html"><![CDATA[<script src="/assets/js/DPlayer.min.js"> </script><script src="/assets/js/APlayer.min.js"> </script><blockquote>
<p>关于coursera无法播放,原因其实很简单,就是视频所在服务器被GFW墙了。</p>
</blockquote>
<hr>
<a id="more"></a>
<p>有的时候,我真的会觉得,墙是不是太密了,使得我们不得不选择越过它。</p>
<p>常见的方法是,我们使用一个在墙外的服务器,他可以帮助我们将coursera的流量通过服务器重新转发,然后再原路返回,避过GFW。</p>
<figure class="image-bubble">
<div class="img-lightbox">
<div class="overlay"></div>
<img src="http://cdn.mmmxcc.cn/blog/20170518/101006060.png?imageView2/0/format/webp/interlace/1/q/75|watermark/2/text/ZmVpeWFuZy5saQ==/font/Y29taWMgc2FucyBtcw==/fontsize/500/fill/IzAzQTlGNA==/dissolve/35/gravity/SouthEast/dx/10/dy/10|imageslim
" alt="mark" title="">
</div>
<div class="image-caption">mark</div>
</figure>
<p>因此,首先要获得服务器端口以及相应的密码,这样服务器才可以帮你转发流量。我自己因为也在coursera看视频,所以自己搭建了一个服务器,分享给大家。<a href="https://ss.91vps.club/" target="_blank" rel="external">https://ss.91vps.club/</a></p>
<p>这里我分别举例展示如何快速观看coursera视频。</p>
<p>–</p>