Finisky Garden

Hugo is a nice static site generator. A common scenario is to store your website source code in a private repository and serve it on GitHub Pages. Can we leverage the github actions to automatically build and deploy the site from the private repository to GitHub Pages? The answer is absolutely yes!

Before we start, you need to have two repos (can belong to different github accounts):

• Source Repo: the repo to store
• Target Repo: host the GitHub Pages (xxx.github.io)

My first understanding of a language model is originated from n-gram. When I know RNNLM, I have a question: why a neural network can represent a language model?

After some research, I found the answer. Essentially, language model is a probability distribution:

A statistical language model is a probability distribution over sequences of words.

最初对语言模型的理解源于n-gram语言模型，但后来出现了RNNLM等一众神经网络语言模型，就有了这个疑问：神经网络为什么可以表示语言模型？

首先，语言模型本质上是概率分布：

A statistical language model is a probability distribution over sequences of words.

成人之美公众号源于高知群体的一个婚恋信息分享群，立足于父母之间的信息共享帮助子女找到合适的另一半。我们定期发布身边靠谱优质的男生女生的资源，如果您也为身边单身的TA感到着急，快快参考我们的发布指南和应征指南吧！

# Scaling Laws for Neural Language Models

一篇实验Paper，调研了神经网络语言模型交叉熵损失变化满足power-law定律，挺有意思的文章。Transformer之后有许多探索不同模型结构的文章，并在一些任务上取得了新的SOTA，却鲜有人考虑影响模型性能的主要因素是什么。

Throughout we will observe precise power-law scalings for performance as a function of training time, context length, dataset size, model size, and compute budget.

LeetCode Top K Frequent Words need to use STL priority_queue to solve the problem.

priority_queue definition:

template<
    class T,
    class Container = std::vector<T>,
    class Compare = std::less<typename Container::value_type>
> class priority_queue;

There are two ways to implement the priority_queue compare function of customized type.

LeetCode Top K Frequent Words 中会用到priority_queue，同时需要定义priority_queue的排序算法。

priority_queue的定义:

template<
    class T,
    class Container = std::vector<T>,
    class Compare = std::less<typename Container::value_type>
> class priority_queue;

有两种方式实现自定义类型priority_queue的比较函数。

This blog uses github as image hosting service after migration. However, sometimes the image loading speed is slow. After some investigation, we found that we can easily improve the image loading speed by jsdelivr CDN.

在迁移博客之后，就切换了图床，使用github作免费的图床。但最近发现它不太稳定，常常打不开。研究发现可以用jsdelivr作github的CDN加速，只需替换下图片地址即可。这才是github图床正确的打开方式 :-)

之前我们谈到 # MongoDB事务重试实现. 如果在事务中使用了BulkWrite()，那么这个新的事务API可能会无限重试从而导致服务器CPU使用率100% (MongoDB Server v4.4.6-ent, MongoDB Driver v2.12.2)。

为避免这个问题，有三个客户端实现的建议：

• 事务API传入cancellation token，限制事务的最长执行时间
• 超过最大重试次数后强制退出事务，避免无限重试
• 设置BulkWrite()按序执行

其中前两个建议对所有事务实现都建议使用，可避免极端情况下对服务端造成不必要的负载。