EINDEX's Blog

2

Sat, 18 Oct 2025 05:01:59 GMT

I recently migrated my blog from Ghost to a new Astro-based system using the Pure Theme. A key benefit of this move was consolidating my separate English site and Chinese site into a single platform.

This transition was primarily motivated by the looming expiration of my Digital Ocean VPS. Coupled with Astro's significant maturation over the past two years and some available time while job seeking, it felt like the perfect opportunity to rebuild.

Familiar features like webmentions, security.txt, and comments will be making a comeback. I'm also excited to integrate new functionalities such as llm.txt support and ActivityPub in the future.

Recent Finding

Deepwiki

DeepWiki is an intelligent tool designed to help developers quickly understand open-source code repositories. It can generate documentation-like wiki pages for your GitHub repository with a single click and allows you to ask questions directly about the codebase.

I tested DeepWiki with my Logseq Copilot Chrome extension repository, and it impressively generated diagrams for architecture, logic, and flow, along with highlighting key features. This tool shows great promise for security reviews, including threat modeling, security code reviews, and system audits. Its output could even provide valuable context for tools like STRIDE-GPT to conduct more in-depth analysis.

MCP Security Risks

The linked article highlights that MCP servers often have excessive permissions and capabilities, introducing significant potential security risks.

Getting Started With Embeddings

This article explains the working principles of embedding models.

2024-ending

Thu, 13 Feb 2025 04:34:04 GMT

My 2024 was primarily focused on exploring technical management and starting immigration.

2024 was a year full of changes and rich experiences for me. During this year, I led a functional team for the first time, spent most of my time in Shanghai, left ThoughtWorks after 5.5 years, sold my apartment in Wuhan, and arrived in Melbourne on Christmas Eve, feeling optimistic about my new life.

As usual, I learned and used new tools almost every week, tinkering with various things on my self-hosted cluster. However, due to being extremely busy with work in 2024, I had little time to write articles and summarize my experiences. Combined with the numerous changes throughout the year and being in a constant state of motion, it's now a month and a half into 2025 before I started writing my 2024 review.

Career

Role and Transition

First, let's talk about my career. The main change in 2024 was transitioning from security consulting to becoming a Function Owner, leading the IAM team in China to plan roadmaps, improve maturity, implement various improvements, and support other teams in better and faster integration with IAM infrastructure. Through Standard Operation Process (SOP), we supported the integration of dozens of applications while introducing SCIM for efficient cross-site data synchronization. We also implemented some automation work in Jira to make the entire integration process visible and ensure smooth communication between team members for timely problem-solving and strategy adjustment.

Compared to my previous work experiences, this role required creating various SOPs and dealing with different vendors, making comparisons, and proposing technical selection solutions. Unlike before, I couldn't personally handle every task and had to focus on high-priority and important deliverables, leaving little time for other matters. While some things didn't go as expected, each module still worked very well. I plan to write an article later sharing the solutions and experiences in Enterprise Identity Management System (EIAM).

Communication and Collaboration

The biggest challenge in my new role was communication. As an internal IAM provider, since most B2B products in China have low integration with IAM or none at all, many business or technical stakeholders don't understand the significance and value of integrating IAM.

The role transition brought new responsibilities where I needed to constantly approach from the perspective of other departments' existing or new products, providing suitable, easy, and stable integration methods to help various applications quickly and steadily integrate with the IAM platform.

The most valuable points in cross-team communication were:

Thinking from the application's perspective, saving integration costs for application teams while providing consistent experiences to reduce learning and usage costs for non-technical staff
Providing successful cases from other teams/similar applications and explaining their benefits
Active communication, timely feedback on issues, and providing sufficient solutions with pros and cons for application teams to choose from
For applications that truly cannot integrate, finding ways to provide grace periods, such as developing/preparing features that will make future integration into the project easier

Professional Skills

Although most of my energy in 2024 was focused on IAM-related work, I didn't neglect AppSec and DevOps. I provided many valuable suggestions to vendors regarding IAM-related application security and DevOps, while also participating in numerous open source projects with the help of AI.

My architectural capabilities improved; in communications with different vendors, I could easily understand the underlying mechanisms of their applications and help them fix issues.

I learned how to make trade-offs and choose the most beneficial paths, avoiding the engineering mindset of automating first before considering value. Simply put, if something has value, do it manually first, then automate it once it stabilizes.

The technology I experimented with most in 2024 was AI-related, though mostly limited to configuration and usage without delving into AI fundamentals - essentially using AI as infrastructure. I created an automated flow using AI to analyze articles in my read-later list and tag them after searching my knowledge base.

Projects

Open Source Projects

The only new project in 2024 was Kobo Hoarder Read-it-later, a tool that modifies the read-later feature on Kobo e-readers to work with Wallabag. Recently, I migrated from Wallabag to Hoarder, so I also modified this project into a Kobo Hoarder tool. The Hook part is directly inherited from KoboOmnivoreConverter.

Blog and Website

I updated the Logseq-Copilot official website using vuepress, and traffic increased dramatically - almost 10 times that of my blog site. I might optimize it further when I have time.

For my personal blog, I simply migrated to Ghost. There were frequent downtimes throughout the past year, and I didn't have much time to fix them. The most encouraging thing is that I received some sponsors, which is my main motivation for continuing to maintain Logseq Copilot.

Finance

Due to moving to Melbourne and spending half a year in Shanghai for work, I barely traveled. Considering future expenses, I first predicted my spending patterns in Beancount, calculating that my savings would last for over two happy years at my current consumption level.

Meanwhile, Australia's basic interest rate of 5% is higher than China's 2%, so my after-tax income in Australia is slightly higher than in China. However, the cost of living is also higher than in China, so my current funds are still quite far from FIRE (Financial Independence, Retire Early).

Lessons Learned

The biggest failure of 2024 was letting my girlfriend come to Australia mid-year, resulting in us living apart. Against the backdrop of a poor economy, I trusted the company's relocation plan too much, and we only managed to reunite after six months. If you need to relocate internationally, it's best to move together - when one person is in a foreign country, the other can only offer limited help.

Looking Forward to 2025

The primary goal this year is to find a job that can support myself and my family in Australia; I also want to get an Australian driver's license and own a car.

Technically, I will explore both information security and development fields while maximizing the use of AI to improve efficiency. I'll continue to explore new areas and knowledge.

I hope to maintain a healthy body and mind, go skiing in Australia, and travel to Japan with my girlfriend.

I will try to maintain regular article updates and publish more newsletters.

Conclusion

2024 was a very fulfilling year. With the support of friends and colleagues, 2024 was full of achievements and experiences.

I hope to find a job soon and settle down in Australia in 2025.

#1 Mid-Autumn Festival

Tue, 17 Sep 2024 00:00:00 GMT

import { LinkPreview } from 'astro-pure/advanced'

Although it's the Mid-Autumn Festival, I'm still on a business trip in Shanghai, continuing to be a homebody who doesn't go out. I've made some major decisions recently, I hope everything goes smoothly.

Projects

One week before the holiday, I updated Logseq Copilot with support for the Logseq DB version.

Articles

Generalists (juniors) are usually better at innovation than experts

I actually think there are mid and senior level generalists too. Generalists excel at solving problems across different domains/knowledge areas, often arriving at simpler solutions.

Tools

improving-your-code-review-process-with-semgrep

Tue, 13 Dec 2022 12:55:03 GMT

In this post, I will show you how to set up Semgrep for your project, give examples of common code issues that it can detect, and provide best practices for using Semgrep in code review. By the end, you will have a better understanding of how Semgrep can improve your code review process and make your code more secure, reliable and maintainable.

Introduction to Semgrep and why it is Useful for Code Review

Code review is an essential part of the SDLC(software development live cycle). It helps ensure that the code is correct, maintainable, secure, and follows the agreed-upon standards and conventions. However, manual code review can be time-consuming and error-prone, especially for large codebases with many contributors or long time history.

Enter Semgrep, a tool for statically analyzing code and detecting common issues. Semgrep runs mostly language-agnostic analysis on analysis-friendly IR(intermediate language) which translate by AST(abstract syntax tree) tree. This process will not generate AST tree via your compile project with dependencies like CodeQL. So semgrep can run it quick and easy to create rules to find and fix common problems in your code, such as security vulnerabilities, logical errors, and style violations.

Setting up Semgrep for Your Project

To use Semgrep, you need to install the semgrep command-line tool on your system. The installation process varies depending on your operating system and package manager, but in general, you can use the following command:

pip install semgrep

Once you have installed Semgrep, you can use it to scan your codebase for specific patterns. For example, the following command will search for instances of the strcpy function, which is considered insecure and should not be used:

semgrep --config p/python --pattern 'strcpy(x, y)' path/to/code

The --config option specifies the configuration to use for the scan, and the --pattern option specifies the rule to match. In this case, I are using the p/python configuration, which includes rules for detecting common issues in Python code, and I are looking for occurrences of the strcpy function.

You can also finding more rules on the semgrep registry on your project, there are a lot of open source rules provide from semgrep community, link to registry.

Once you have set up Semgrep and run a scan, you can use the results to identify and fix potential problems in your code. This can save time and effort during code review, as it allows you to automatically detect and address common issues before they are reviewed by your team.

Examples of Common Code Issues that Semgrep Can Detect

Semgrep can detect a wide range of code issues, depending on the configuration and rules that you use. Here are some examples of common code issues that Semgrep can help you find and fix:

Security vulnerabilities, such as SQL injection and Cross-Site Scripting (XSS) vulnerabilities, or you can use OWASP Top 10 ruleset for common web security vulnerabilities.
Deprecated or unsafe API usage, such as the use of strcpy in C or eval in JavaScript
Misconfgration, such as dockerfile, docker-compose settings.
Hard-coded secret, such as the secret ruleset.
Style violations, such as inconsistent indentation, unused variables, catch default issues and overly complex expressions
Permission issue, such as forgot add permission check. Need to write custom rules.

By using Semgrep to detect these issues, you can prevent bugs and security risks in your code and improve its overall quality. This can save time and effort during code review, as it allows you to focus on higher-level design and architecture issues instead of catching low-level mistakes.

In the next section, I will share some best practices when I using Semgrep in code review.

or create your own configuration and rules to customize Semgrep for your project. For more information, see the Semgrep documentation, link to docs.

Best Practices for Using Semgrep in Code Review

To get the most out of Semgrep in code review, you should follow these best practices:

How to Integrate Semgrep into Your Code Review Workflow

Set up Semgrep as part of your continuous integration (CI) pipeline, so that it automatically runs on every code change. This can help you catch potential issues early and prevent them from being merged into the main codebase.
Use the output of Semgrep as input for your code review tool, such as GitHub Pull Requests or Gerrit. This can help you quickly identify and address the issues that Semgrep has detected.
Use Semgrep as part of your team's code review standards and guidelines. This can help your team members understand the expectations for code quality and avoid common pitfalls.
Starting to write costom rules for your project. This will building you team knowledge base for your codebases, new team member can easily and fast know the reason, and avoid issues.

How to Write effective Semgrep Rules

Keep your rules simple and focused: avoid using complex regular expressions or matching patterns that are too broad.
Use concrete examples to illustrate the code that the rule should match and the code that it should not match. This can help you and others understand the rule and avoid false positives or false negatives.
Test your rules on a representative sample of your codebase before using them in code review. This can help you fine-tune the rules and make sure they work as expected.
If possible provide fix suggestion or direct using fix field in the rule for time saving.
Document your rules and share them with your team. This can help others understand the rationale behind the rules and improve the consistency of your codebase.

By following these best practices, you can make Semgrep an integral part of your code review process and improve the overall quality of your code. In the next section, I will discuss some tips and tricks for getting the most out of Semgrep.

Tips and Tricks for Getting the Most out of Semgrep

To get the most out of Semgrep, here are some tips and tricks that you can use:

Use the --severity option to filter the output of Semgrep based on the severity of the issues that it detects. This can help you focus on the most important or critical issues first.
Use the --metrics off or set environment variables SEMGREP_SEND_METRICS=off if you not want to send anything to Semgrep server, avoiding the information exposure.
Use the --json, --junit-xml or --sarif as output, can easily to integrate your workflow.
Use the --autofix with simple code style ruleset as lint, run it in your team member's pre-commit hook.

By using these tips and tricks, you can customize Semgrep to fit your specific needs and workflows, and get even more value from it. In the next and final section, I will discuss the conclusion and next steps for improving your code review process with Semgrep.

Conclusion and next Steps for Improving Your Code Review Process with Semgrep

In this post, I have shown you how to use Semgrep to improve your code review process. I have discussed the benefits of using Semgrep, the steps for setting it up, the types of code issues that it can detect, and the best practices for using it in code review. I have also provided some tips and tricks for getting the most out of Semgrep.

By using Semgrep in your code review process, you can save time and effort, improve the quality of your code, and prevent bugs and security risks. If you want to learn more about Semgrep and how it can help you, here are some next steps that you can take:

Read the Semgrep documentation to learn more about the features and capabilities of Semgrep, link to docs.
Try Semgrep on your own codebase to see how it works and what it can find.
Learning how to write semgrep rules on official tutorial, link to learning center.
Join the Semgrep community to ask questions, share experiences, and contribute to the development of Semgrep, Slack, Twitter.

I hope that this post has been helpful and that you will consider using Semgrep in your code review process. Happy coding!

webmentions

Sun, 06 Nov 2022 09:44:50 GMT

Recently I came across Now, I'm in IndieWeb? and learned about webmentions. I found the concept very interesting and decided to join the webmentions community for my blog.

What Are Webmentions

Webmentions are a way for websites to communicate with each other and notify each other when they link to each other's content. This allows for a more interactive and connected online community, where websites can engage with each other and their users in real-time.

How Do I Join the Webmentions Community?

To join the webmentions community and start receiving notifications when other websites link to your content, you need to take the following steps:

Register your website with a webmention service, such as webmention.io. This will allow your website to receive webmentions from other sites and also enable you to send webmentions to other sites when you link to them. To do this, you can add the following HTML code to your website:
```
<link rel="webmention" href="https://webmention.io/eindex.me/webmention" />
<link rel="pingback" href="https://webmention.io/eindex.me/xmlrpc" />
```
Make sure your website follows the microformats2 standard. This may require some effort, but it is important for ensuring that your website can properly send and receive webmentions. You can use a tool like IndieWebify.Me to check if your website is using microformats2 correctly. You may also want to use a tool like ngrok or cloudflare tunnel to debug your website locally, which can save you time.
Start sending and receiving webmentions on your website. Once your website is registered with a webmention service and you have tested your implementation, you can start using webmentions on your site. Whenever you link to another website, you can send a webmention to notify them of your link. And whenever someone else links to your website, you can receive a webmention notification and display that link on your site.
Consider using tools like Github actions and JavaScript scripts to automate the process of sending and receiving webmentions. This is particularly useful for static websites. For example, you can store your webmentions in a webmentions branch on Github, and use a workflow file like this to fetch webmentions and send your webmentions, you also can check my implement for this blog:
```
name: Fetch Webmentions
on:
schedule:
    - cron: "0 0 * * *"
jobs:
fetch_webmentions:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
        with:
        ref: webmentions
    - name: Fetch Webmentions
        run: node pipeline/webmentions.js
    - name: Send Webmetnions
        run: node pipeline/send-webmentions.js
```

By following these steps, you can join the webmentions community and start enjoying the benefits of more interactive and engaging communication with other websites and their users.

Why Join the Webmentions Community?

There are many potential benefits to joining the webmentions community and implementing webmentions on your website. Some of the potential benefits include:

Increased engagement and connectivity with other websites and theirusers. By receiving notifications when other websites link to your content and being able to notify other websites when you link to them, you can foster a more interactive and connected online community. This can lead to increased traffic and potentially even more opportunities for collaboration and engagement.
Improved communication and collaboration with other websites. Webmentions allow for real-time communication between websites, which can be helpful for coordinating efforts and staying up-to-date with the latest developments in the webmentions community.
The ability to participate in the evolving web ecosystem. Webmentions are a form of "web 3.0" where websites are more interconnected and interactive than in the past. By joining the webmention community, you can be part of this evolving web ecosystem and potentially benefit from new opportunities and connections that arise.

Of course, joining the webmention community does require some effort and resources, so it may not be the right decision for everyone. But for many website owners, the potential benefits of joining the webmention community may outweigh the challenges, making it a worthwhile investment.

Conclusion

In this beginner's guide, we've covered the steps involved in joining the webmentions community and implementing webmentions on your website. By following these steps, you can join the webmentions community and start enjoying the benefits of more interactive and engaging communication with other websites and their users. While there are some challenges and resources required to join the webmention community, the potential benefits may outweigh the challenges for many website owners.

semgrep-with-custom-rules

Wed, 14 Dec 2022 06:14:56 GMT

In this post, we will discuss how Semgrep can be integrated into your code review process. If you are unfamiliar with Semgrep, we recommend checking out our previous article on the topic, titled "Improving Your Code Review Process With Semgrep".

Codereview and Semgrep

Code review is a crucial step in the software development workflow, as it allows team members to review and critique each other's code.

However, code review can sometimes be a source of contention, with debates over code style or disagreements over team coding standards.

Additionally, there may be cases where the team lacks domain expertise, leading to the inclusion of vulnerabilities in the code. Semgrep can help address these issues by promoting consistent coding practices and identifying potential vulnerabilities.

Let's take a closer look at how Semgrep can be utilized in the code review process.

Cases

In this section, we will examine three cases where Semgrep custom rules can be utilized to improve code quality.

Case 1: Preventing the use of System.out in Java. It is not uncommon for team members to use System.out for logging or debugging purposes. However, this practice can be problematic. The following custom rule can help address this issue:

rules:
  - id: logging-via-system-out
    pattern: System.out.$PRINT($ARG);
    message: Using system out as logging method.
    languages: [java]
    severity: WARNING

If your team has established logging standards, the fix function can be used to automatically replace instances of System.out with a preferred logging method. For example:

rules:
  - id: logging-via-system-out
    pattern: System.out.$PRINT($ARG);
    message: Using system out as logging method.
    fix: log.info($ALG)
    languages: [java]
    severity: WARNING

Case 2: Disallowing system calls. In some situations, it may be desirable to prohibit system commands in order to protect the user's device or cloud machine. The following custom rule can be used to detect and prevent the execution of system commands:

rules:
  - id: java-rec-checker
    patterns:
      - pattern-inside: $FUN(...,$ARG,...){...}
      - pattern-either:
          - pattern: Runtime.getRuntime().exec(..., $ARG, ...);
          - patterns:
              - pattern: Process $PROCESS = Runtime.getRuntime();
              - pattern: $PROCESS.exec(..., $ARG, ...);
    message: RCE risk from user input.
    languages: [java]
    severity: ERROR

Case 3: Ensuring that all request mappings have authentication checks. It is important to ensure that all request mappings have proper permission checks in place. The following custom rule can be used to identify instances where these checks are missing:

rules:
  - id: vaild-permission-check-on-all-request-mapping
    patterns:
      - pattern-inside: |
          @$CONTROLLER_ANNOTATION
          public class $CONTROLLER {
            ...
          }
      - pattern: |
          @$MAPPING_ANNOTATION(...)
          public $RET $METHOD(...){...}
      - pattern-not: |
          @$MAPPING_ANNOTATION(...)
          @$PERMISSION_ANNOTATION(...)
          public $RET $METHOD(...){...}
      - metavariable-regex:
          metavariable: $CONTROLLER_ANNOTATION
          regex: (.*Controller$)
      - metavariable-regex:
          metavariable: $MAPPING_ANNOTATION
          regex: (.*Mapping$)
      - metavariable-regex:
          metavariable: $PERMISSION_ANNOTATION
          regex: (.*Permission$)
    message: Should check user permission on all request mapping
    languages: [java]
    severity: WARNING

Summarize

This article has discussed how Semgrep can be integrated into the code review process to improve code quality and consistency.

By using custom rules, Semgrep can enforce team coding standards and detect potential vulnerabilities in the code.

By incorporating Semgrep into the code review workflow, teams can more effectively identify and address issues before code is released to production.