Today I found that some pods in kubernetes cluster are failed, the status is Waiting: ContainerCreating. The pod events:

1
2
3
4

MountVolume.SetUp failed for volume "xxxxx" : secret "xxxxx" not found
kubelet aks-agentpool-xxx-vmss000001

Unable to attach or mount volumes: unmounted volumes=[xxxxx], unattached volumes=[xxxxx]: timed out waiting for the condition

I remember that about one week ago I delete some secretes in this cluster. Therefore, the problem becomes how to recover the deleted secret “xxxxx”?

Recently, I found there were huge amount of files named .com.google.Chrome.* be created in my /tmp folder. Obviously the culprit is Chrome. However, after some research no solution is found to prevent Chrome creating these garbage.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

/tmp$ du -csh .com.google.Chrome.*

8.0K    .com.google.Chrome.00OKwD
104K    .com.google.Chrome.013jYf
172K    .com.google.Chrome.015x5t
...
48K     .com.google.Chrome.Zytrhf
16K     .com.google.Chrome.zz233G
36K     .com.google.Chrome.ZzrsZY
163M    total
/tmp$  find /tmp -name ".com.google.Chrome*" -ls| wc -l
3468

Update 2020/12/12 I found there are lots of .com.google.Chrome.* files in /tmp/snap.chromium/tmp :-( . Look at the ncdu results:

Yesterday, when I login to my VM I found that the disk is full. Even a tab command completion cannot be done. I immediately delete some unused files and the system works. However, when I reboot the system, it stucked at A start job is running for Create Volatile Files and Directories...:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

...
[  OK  ] Started File System Check on /dev/disk/cloud/azure_resource-part1.
[  OK  ] Started File System Check Daemon to report status.
[  OK  ] Started File System Check on /dev/d…5d996-7436-4ec8-b5d6-0d7b6100aeb5.
         Mounting /data...
[  OK  ] Mounted /data.
[  OK  ] Reached target Local File Systems.
         Starting AppArmor initialization...
         Starting ebtables ruleset management...
         Starting Tell Plymouth To Write Out Runtime Data...
         Starting Set console font and keymap...
[  OK  ] Started Set console font and keymap.
[  OK  ] Started Tell Plymouth To Write Out Runtime Data.
[  OK  ] Started ebtables ruleset management.
[  OK  ] Started Flush Journal to Persistent Storage.
         Starting Create Volatile Files and Directories...
[  OK  ] Started AppArmor initialization.
[    **] A start job is running for Create V… Directories (13min 46s / no limit)

I searched and found a similar discussion: # Boot stuck at “A start job is running for Create Volatile Files and Directories”

After I changed the HOME folder to another place, I copied the ssh config folder from old HOME to the new place. Supposedly it should directly work, right? However, when I login the server with my private key, the server said: “Server Refused Our Key”…

I spent some time to figure out the problem: new HOME folder access mode issue, it SHOULD NOT have write access for group.

Unity supports three ways of registering types:

• Instance registration
• Type registration
• Factory registration

Typically, Instance registration and Type registration resolve dependencies through ResolvedParameter<T>, while Factory registration resolve dependencies by a factory delegate. In practice, when you want to resolve a List<T> or Dictionary<T1, T2>, Factory registration is what you want. Let’s go through how to resolve a collection of customized classes.

Animal Example

Start with example, assume we have an IAnimal interface which has two implementations. There is a Zoo class which accept List<IAnimal> as the parameter:

If you use Visual Studio 2017 and create an .NET Core projects the first time, the following errors might appear:

The current .NET SDK does not support targeting .NET Standard 2.0. Either target .NET Standard 1.6 or lower, or use a version of the .NET SDK that supports .NET Standard 2.0. XXXProject C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\Sdks\Microsoft.NET.Sdk\build\Microsoft.NET.TargetFrameworkInference.targets

Another related issue is .NET Core project load failed. Both errors point out that the corresponding .NET SDK is missing.

Recently we use Grafana to monitor ASP.NET Core apps. We have an interesting observation that sometimes the “allocated memory” is larger than “working set”. After investigation, we found the root cause is that the app uses native dlls which operates on unmanaged memory which is not managed by GC. Therefore, how to correctly collect memory related metrics in C#?

We will explain these related concepts in this post.

Basic Concepts

Managed vs Unmanaged Code

# What is “managed code”?

The error looks like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

$ python test.py
Traceback (most recent call last):
  File "test.py", line 3, in <module>
  ...
  File "miniconda3/envs/torch/lib/python3.7/site-packages/apex/__init__.py", line 18, in <module>
    from apex.interfaces import (ApexImplementation,
  File "miniconda3/envs/torch/lib/python3.7/site-packages/apex/interfaces.py", line 10, in <module>
    class ApexImplementation(object):
  File "miniconda3/envs/torch/lib/python3.7/site-packages/apex/interfaces.py", line 14, in ApexImplementation
    implements(IApex)
  File "miniconda3/envs/torch/lib/python3.7/site-packages/zope/interface/declarations.py", line 483, in implements
    raise TypeError(_ADVICE_ERROR % 'implementer')
TypeError: Class advice impossible in Python3.  Use the @implementer class decorator instead.

Refer to this issue: https://github.com/NVIDIA/apex/issues/116

A weird problem that execute “import torch” in bash works but when you run it in Jupyter notebook:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

ImportError Traceback (most recent call last)
<ipython-input-4-8ba1970b60ce> in <module>
 6 import random
 7 
----> 8  import torch
 9 import torch.nn as nn
 10 

~/miniconda3/envs/tf/lib/python3.6/site-packages/torch/__init__.py in <module>
 79 del _dl_flags
 80 
---> 81  from torch._C import *
 82 
 83 __all__ += [name for name in dir(_C)

ImportError: dlopen: cannot load any more object with static TLS

Seems that it’s a compatibility issue. Even though there are a large volume of discussions but none of them works. Ironically, I accidently fixed the issue by resolving another: # RuntimeError cuDNN error CUDNN_STATUS_EXECUTION_FAILED Solution

1

RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED

When you encountered the above issue and Google it, you will find lots of discussions. Unfortunately, very rare of them are useful and work.

Actually, the root cause is pytorch/cuda/python compatibility issue.

Solution

The solution is straightforward. Simply downgrade pytorch and install a different version of cuda or python would be fine.

My environment:

1. Ubuntu 18.04 LTS
2. Python 3.6.9
3. PyTorch 1.3.0
4. cuda 10.1

This command resolved my issue (PyTorch version really matters! from 1.3.0 to 1.2.0):

When I use kubectl to deploy a service:

1

$ kubectl --kubeconfig=C:\Users\xxx\.kube\config apply -f deploy.yaml --namespace=xxx

a SchemeError was raised:

1

EXEC(0,0): Error : SchemaError(io.k8s.api.apps.v1beta1.RollingUpdateStatefulSetStrategy): invalid object doesn't have additional properties

Seems that the issue is caused by kubectl version mismatch: Install and Set Up kubectl

1

You must use a kubectl version that is within one minor version difference of your cluster. For example, a v1.2 client should work with v1.1, v1.2, and v1.3 master. Using the latest version of kubectl helps avoid unforeseen issues.

Follow these steps to resolve the issue:

I have encountered an issue: solution can be successfully built and run, but lots of classes in the VisualStudio are marked as red color by ReSharper. Clean and rebuild the solution doesn’t work. Something like this: Everything compiles, but Resharper marks everything in red?

Clean ReSharper Cache

The following instructions are tested under Resharper Ultimate 2019.2:

Hexo 4.0.0 has been released on 2019-10-14. How to upgrade it?

Upgrade Hexo

1. Go to the hexo folder
2. Modify package.json file, inside “dependencies” braces:

1
2

-    "hexo": "^3.9.0",
+    "hexo": "^4.0.0",

UNNECESSARY to change

1
2
3

"hexo": {
    "version": "3.9.0"
},

and it will be automatically updated after “npm update”.

3. Run

1

$ npm update

Done. Then check the hexo version as below.

When you use RegisterType or RegisterInstance combined with InjectionFactory in old version of Unity, the following complier warning will be displayed:

‘InjectionFactory’ is obsolete: ‘InjectionFactory has been deprecated and will be removed in next release. Please use IUnityContainer.RegisterFactory(…) method instead.

The code that triggered the warning:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

var container = new UnityContainer();
container.RegisterType<IAnimal, Cat>("Cat", new InjectionConstructor());
container.RegisterType<IAnimal, Dog>("Dog", new InjectionConstructor());

container.RegisterType<List<IAnimal>>(
    "AnimalList",
    new InjectionFactory(
        m => new List<IAnimal>
        {
            m.Resolve<IAnimal>("Cat"),
            m.Resolve<IAnimal>("Dog"),
        }
        )
);

The solution is straightforward, follow the instruction and use RegisterFactory instead:

Deploy Azure cloudservice failed, and Microsoft Azure Activity Log shows the following message：

1
2
3
4

15:45:39 - Checking for Remote Desktop certificate...  
15:45:40 - Applying Diagnostics extension.  
15:46:03 - Applying remote desktop protocol (RDP) extension.  
15:46:03 - Keyset does not exist.

It seems that the problem is caused by the certificate which is used to encrypt the user credentials of RDP.

First, I found the corresponding certificate in ServiceConfiguration.Cloud.cscfg (hiding the thumbprint):

Intro to NDCG

Normalized Discounted Cumulative Gain (NDCG) is a measure of ranking quality. Typically, it is used to measure the performance of a ranker and widely adopted in information retrieval. Our goal is to rank relevant documents higher than irrelavant documents. Here comes the problem, given a list of documents with corresponding grades, how to model the ranking quality?

Concretely, assume we want to rank a list of 5 documents with grades: $$G = [3, 1, 2, 0, 2]$$

Description

After deploying the Hexo websites to server, all the NexT icons were shown as blank squares.

Analysis

The NexT theme uses Font Awesome to display icons. If the website is started locally, everything is fine. Another interesting point is that the issue exists in Chrome but not IE.

1. Stylish plugin in Chrome. I forced all pages use customized fonts by Stylish. Maybe the font does not contain such icons. However, even I disable the plugin, the issue still exists.