{"id":6137,"date":"2024-06-14T10:22:18","date_gmt":"2024-06-14T02:22:18","guid":{"rendered":"https:\/\/t.n-years.com\/?p=6137"},"modified":"2024-06-14T10:22:19","modified_gmt":"2024-06-14T02:22:19","slug":"%e6%89%a9%e6%95%a3%e6%a8%a1%e5%9e%8b","status":"publish","type":"post","link":"https:\/\/t.n-years.com\/?p=6137","title":{"rendered":"\u6269\u6563\u6a21\u578b"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">\u975e\u539f\u521b\uff0cChatGPT \u95ee\u7b54\u751f\u6210<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u6269\u6563\u6a21\u578b\uff08Diffusion Models\uff09\u662f\u673a\u5668\u5b66\u4e60\u9886\u57df\u4e2d\u7684\u4e00\u79cd\u751f\u6210\u6a21\u578b\uff0c\u5b83\u4eec\u901a\u8fc7\u6a21\u62df\u6570\u636e\u4ece\u7b80\u5355\u5206\u5e03\u5411\u590d\u6742\u5206\u5e03\u7684\u6f14\u5316\u8fc7\u7a0b\u6765\u751f\u6210\u6570\u636e\u3002\u8fd9\u4e9b\u6a21\u578b\u6700\u8fd1\u5728\u56fe\u50cf\u751f\u6210\u548c\u5176\u4ed6\u751f\u6210\u4efb\u52a1\u4e2d\u8868\u73b0\u51fa\u8272\u3002\u4e0b\u9762\u662f\u5bf9\u6269\u6563\u6a21\u578b\u7684\u8be6\u7ec6\u4ecb\u7ecd\uff1a<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">\u57fa\u672c\u6982\u5ff5<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">\u6269\u6563\u6a21\u578b\u7684\u6838\u5fc3\u601d\u60f3\u662f\u901a\u8fc7\u9010\u6b65\u6dfb\u52a0\u566a\u58f0\u5c06\u6570\u636e\u4ece\u590d\u6742\u5206\u5e03\uff08\u5982\u771f\u5b9e\u56fe\u50cf\u5206\u5e03\uff09\u53d8\u6362\u5230\u7b80\u5355\u5206\u5e03\uff08\u5982\u9ad8\u65af\u5206\u5e03\uff09\uff0c\u7136\u540e\u5b66\u4e60\u5982\u4f55\u9006\u5411\u64cd\u4f5c\uff0c\u5373\u4ece\u7b80\u5355\u5206\u5e03\u9010\u6b65\u53bb\u566a\u91cd\u5efa\u590d\u6742\u5206\u5e03\u7684\u6570\u636e\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">\u4e3b\u8981\u6b65\u9aa4<\/h3>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>\u6b63\u5411\u6269\u6563\u8fc7\u7a0b\uff08Forward Diffusion Process\uff09<\/strong>\uff1a<\/li>\n<\/ol>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u901a\u8fc7\u591a\u4e2a\u6b65\u9aa4\u9010\u6e10\u5411\u6570\u636e\u6dfb\u52a0\u566a\u58f0\uff0c\u4f7f\u5176\u6700\u7ec8\u63a5\u8fd1\u4e00\u4e2a\u7b80\u5355\u7684\u5206\u5e03\uff08\u5982\u6807\u51c6\u9ad8\u65af\u5206\u5e03\uff09\u3002<\/li>\n\n\n\n<li>\u6bcf\u4e00\u6b65\u90fd\u6dfb\u52a0\u4e00\u5b9a\u91cf\u7684\u9ad8\u65af\u566a\u58f0\uff0c\u5f62\u6210\u4e00\u4e2a\u9010\u6b65\u6076\u5316\u7684\u5e8f\u5217\u3002<\/li>\n<\/ul>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>\u9006\u5411\u6269\u6563\u8fc7\u7a0b\uff08Reverse Diffusion Process\uff09<\/strong>\uff1a<\/li>\n<\/ol>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u5b66\u4e60\u5982\u4f55\u9010\u6b65\u53bb\u566a\uff0c\u4ece\u566a\u58f0\u6570\u636e\u6062\u590d\u539f\u59cb\u6570\u636e\u3002<\/li>\n\n\n\n<li>\u8fd9\u4e2a\u8fc7\u7a0b\u901a\u8fc7\u8bad\u7ec3\u4e00\u4e2a\u795e\u7ecf\u7f51\u7edc\u6a21\u578b\u6765\u5b9e\u73b0\uff0c\u8be5\u6a21\u578b\u5728\u6bcf\u4e2a\u6b65\u9aa4\u90fd\u5c1d\u8bd5\u53bb\u9664\u6dfb\u52a0\u7684\u566a\u58f0\u3002<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">\u6570\u5b66\u516c\u5f0f<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">\u8bbe ( x_0 ) \u662f\u539f\u59cb\u6570\u636e\uff0c\u6b63\u5411\u6269\u6563\u8fc7\u7a0b\u4ea7\u751f\u4e00\u7cfb\u5217\u6570\u636e ( x_1, x_2, \\ldots, x_T )\uff0c\u5176\u4e2d ( T ) \u662f\u6269\u6563\u6b65\u9aa4\u6570\u3002\u6bcf\u4e00\u6b65\u7684\u6b63\u5411\u8fc7\u7a0b\u5b9a\u4e49\u4e3a\uff1a<br>[ q(x_t | x_{t-1}) = \\mathcal{N}(x_t; \\sqrt{\\alpha_t} x_{t-1}, (1 - \\alpha_t) I) ]<br>\u5176\u4e2d\uff0c ( \\alpha_t ) \u662f\u63a7\u5236\u566a\u58f0\u5f3a\u5ea6\u7684\u53c2\u6570\u3002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u9006\u5411\u8fc7\u7a0b\u7684\u76ee\u6807\u662f\u5b66\u4e60 ( p_\\theta(x_{t-1} | x_t) )\uff0c\u53ef\u4ee5\u8fd1\u4f3c\u4e3a\uff1a<br>[ p_\\theta(x_{t-1} | x_t) = \\mathcal{N}(x_{t-1}; \\mu_\\theta(x_t, t), \\Sigma_\\theta(x_t, t)) ]<br>\u795e\u7ecf\u7f51\u7edc ( \\mu_\\theta ) \u548c ( \\Sigma_\\theta ) \u7528\u4e8e\u9884\u6d4b\u5747\u503c\u548c\u65b9\u5dee\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">\u6a21\u578b\u8bad\u7ec3<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">\u6a21\u578b\u7684\u8bad\u7ec3\u76ee\u6807\u662f\u6700\u5c0f\u5316\u771f\u5b9e\u6b63\u5411\u8fc7\u7a0b ( q ) \u548c\u6a21\u578b\u9006\u5411\u8fc7\u7a0b ( p_\\theta ) \u4e4b\u95f4\u7684Kullback-Leibler\u6563\u5ea6\uff08KL\u6563\u5ea6\uff09\uff0c\u5177\u4f53\u516c\u5f0f\u4e3a\uff1a<br>[ L(\\theta) = \\mathbb{E}<em>q \\left[ \\sum<\/em>{t=1}^T D_{KL}(q(x_{t-1} | x_t, x_0) | p_\\theta(x_{t-1} | x_t)) \\right] ]<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u5728\u5b9e\u8df5\u4e2d\uff0c\u5e38\u7528\u7b80\u5316\u7684\u635f\u5931\u51fd\u6570\uff0c\u4f8b\u5982\u9884\u6d4b\u566a\u58f0\u7684\u65b9\u6cd5\u6765\u8bad\u7ec3\u6a21\u578b\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">\u4f18\u70b9\u548c\u5e94\u7528<\/h3>\n\n\n\n<h4 class=\"wp-block-heading\">\u4f18\u70b9<\/h4>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>\u751f\u6210\u8d28\u91cf\u9ad8<\/strong>\uff1a\u5728\u56fe\u50cf\u751f\u6210\u4efb\u52a1\u4e2d\uff0c\u6269\u6563\u6a21\u578b\u53ef\u4ee5\u751f\u6210\u9ad8\u8d28\u91cf\u548c\u9ad8\u5206\u8fa8\u7387\u7684\u56fe\u50cf\u3002<\/li>\n\n\n\n<li><strong>\u591a\u6837\u6027\u597d<\/strong>\uff1a\u7531\u4e8e\u6269\u6563\u8fc7\u7a0b\u7684\u968f\u673a\u6027\uff0c\u751f\u6210\u7684\u6837\u672c\u5177\u6709\u8f83\u597d\u7684\u591a\u6837\u6027\u3002<\/li>\n\n\n\n<li><strong>\u7a33\u5b9a\u6027\u5f3a<\/strong>\uff1a\u76f8\u6bd4\u5176\u4ed6\u751f\u6210\u6a21\u578b\uff0c\u6269\u6563\u6a21\u578b\u5728\u8bad\u7ec3\u8fc7\u7a0b\u4e2d\u8f83\u4e3a\u7a33\u5b9a\u3002<\/li>\n<\/ol>\n\n\n\n<h4 class=\"wp-block-heading\">\u5e94\u7528<\/h4>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>\u56fe\u50cf\u751f\u6210<\/strong>\uff1a\u5982DALL-E 2\u3001Stable Diffusion\u7b49\uff0c\u7528\u4e8e\u751f\u6210\u9ad8\u8d28\u91cf\u7684\u56fe\u50cf\u3002<\/li>\n\n\n\n<li><strong>\u6570\u636e\u53bb\u566a<\/strong>\uff1a\u901a\u8fc7\u9006\u5411\u8fc7\u7a0b\u53bb\u566a\uff0c\u5e94\u7528\u4e8e\u56fe\u50cf\u53bb\u566a\u3001\u56fe\u50cf\u4fee\u590d\u7b49\u3002<\/li>\n\n\n\n<li><strong>\u5176\u4ed6\u751f\u6210\u4efb\u52a1<\/strong>\uff1a\u6587\u672c\u751f\u6210\u3001\u97f3\u9891\u751f\u6210\u7b49\u3002<\/li>\n<\/ol>\n\n\n\n<h3 class=\"wp-block-heading\">\u793a\u4f8b<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">\u4ee5\u56fe\u50cf\u751f\u6210\u4efb\u52a1\u4e3a\u4f8b\uff0c\u5047\u8bbe\u6211\u4eec\u6709\u4e00\u4e2a\u8bad\u7ec3\u96c6\u5305\u542b\u5927\u91cf\u771f\u5b9e\u56fe\u50cf\u3002\u6211\u4eec\u53ef\u4ee5\u6309\u7167\u4ee5\u4e0b\u6b65\u9aa4\u4f7f\u7528\u6269\u6563\u6a21\u578b\u8fdb\u884c\u56fe\u50cf\u751f\u6210\uff1a<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>\u6784\u5efa\u6b63\u5411\u6269\u6563\u8fc7\u7a0b<\/strong>\uff1a\u9010\u6b65\u5411\u56fe\u50cf\u6dfb\u52a0\u9ad8\u65af\u566a\u58f0\uff0c\u76f4\u5230\u56fe\u50cf\u53d8\u4e3a\u7eaf\u566a\u58f0\u3002<\/li>\n\n\n\n<li><strong>\u8bad\u7ec3\u6a21\u578b<\/strong>\uff1a\u4f7f\u7528\u5e26\u566a\u56fe\u50cf\u5e8f\u5217\u8bad\u7ec3\u795e\u7ecf\u7f51\u7edc\u6a21\u578b\uff0c\u5b66\u4e60\u53bb\u566a\u8fc7\u7a0b\u3002<\/li>\n\n\n\n<li><strong>\u751f\u6210\u56fe\u50cf<\/strong>\uff1a\u4ece\u566a\u58f0\u56fe\u50cf\u5f00\u59cb\uff0c\u4f7f\u7528\u8bad\u7ec3\u597d\u7684\u6a21\u578b\u9010\u6b65\u53bb\u566a\uff0c\u751f\u6210\u65b0\u56fe\u50cf\u3002<\/li>\n<\/ol>\n\n\n\n<h3 class=\"wp-block-heading\">\u793a\u4f8b\u4ee3\u7801<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">\u4ee5\u4e0b\u662f\u4e00\u4e2a\u7b80\u5355\u7684\u4f2a\u4ee3\u7801\u793a\u4f8b\uff0c\u5c55\u793a\u5982\u4f55\u8bad\u7ec3\u6269\u6563\u6a21\u578b\uff1a<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>import torch\nimport torch.nn as nn\nimport torch.optim as optim\n\nclass DiffusionModel(nn.Module):\n    def __init__(self):\n        super(DiffusionModel, self).__init__()\n        self.model = nn.Sequential(\n            nn.Conv2d(3, 64, kernel_size=3, padding=1),\n            nn.ReLU(),\n            nn.Conv2d(64, 3, kernel_size=3, padding=1)\n        )\n\n    def forward(self, x, t):\n        return self.model(x)\n\ndef train_diffusion_model(data_loader, model, num_steps, optimizer):\n    for epoch in range(num_epochs):\n        for images in data_loader:\n            optimizer.zero_grad()\n            noisy_images = images + torch.randn_like(images) * (1.0 \/ num_steps)\n            predicted_images = model(noisy_images, t)\n            loss = nn.MSELoss()(predicted_images, images)\n            loss.backward()\n            optimizer.step()\n\n# \u793a\u4f8b\u7528\u6cd5\ndata_loader = ...\nmodel = DiffusionModel()\noptimizer = optim.Adam(model.parameters(), lr=1e-4)\ntrain_diffusion_model(data_loader, model, num_steps=1000, optimizer)<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">\u7ed3\u8bba<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">\u6269\u6563\u6a21\u578b\u901a\u8fc7\u6a21\u62df\u6570\u636e\u4ece\u7b80\u5355\u5206\u5e03\u5230\u590d\u6742\u5206\u5e03\u7684\u9010\u6b65\u6f14\u5316\u8fc7\u7a0b\uff0c\u5b9e\u73b0\u4e86\u9ad8\u8d28\u91cf\u7684\u6570\u636e\u751f\u6210\u3002\u5176\u5728\u56fe\u50cf\u751f\u6210\u7b49\u9886\u57df\u5c55\u73b0\u51fa\u5f3a\u5927\u7684\u80fd\u529b\uff0c\u662f\u73b0\u4ee3\u751f\u6210\u6a21\u578b\u4e2d\u7684\u4e00\u4e2a\u91cd\u8981\u5206\u652f\u3002<\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u975e\u539f\u521b\uff0cChatGPT \u95ee\u7b54\u751f\u6210 \u6269\u6563\u6a21\u578b\uff08Diffusion Models\uff09\u662f\u673a\u5668\u5b66\u4e60\u9886\u57df\u4e2d\u7684\u4e00\u79cd\u751f\u6210\u6a21\u578b\uff0c&#8230; <\/p>\n<div class=\"read-more navbutton\"><a href=\"https:\/\/t.n-years.com\/?p=6137\">\u9605\u8bfb\u66f4\u591a<i class=\"fa fa-angle-double-right\"><\/i><\/a><\/div>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5],"tags":[69,51],"class_list":["post-6137","post","type-post","status-publish","format-standard","hentry","category-5","tag-diffusion","tag-51"],"_links":{"self":[{"href":"https:\/\/t.n-years.com\/index.php?rest_route=\/wp\/v2\/posts\/6137","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/t.n-years.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/t.n-years.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/t.n-years.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/t.n-years.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=6137"}],"version-history":[{"count":1,"href":"https:\/\/t.n-years.com\/index.php?rest_route=\/wp\/v2\/posts\/6137\/revisions"}],"predecessor-version":[{"id":6138,"href":"https:\/\/t.n-years.com\/index.php?rest_route=\/wp\/v2\/posts\/6137\/revisions\/6138"}],"wp:attachment":[{"href":"https:\/\/t.n-years.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=6137"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/t.n-years.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=6137"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/t.n-years.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=6137"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}