Utf-8 Questions

I have an email subject of the form: =?utf-8?b?t3.....?= the body of the email is utf-8 base64 encoded - and has

I'm saving data to firebase rtdb through js sdk (6.3.1) using: firebase.initializeapp(firebaseconfig); var data={

I'm calling the firebase rest api from a node.js process. the problem i'm seeing is that posts fail when the post body contains non-ascii

I'm trying to build a web service using ruby on rails. users authenticate themselves via http basic auth. i want to allow any valid utf-8

Sometimes i have unwanted characters in my files (db tables, git content, etc). when i list them with xxd i have the utf-8 code,

I am trying to convert a huge csv file from utf-16 to utf-8 format using python and below is the code: with

I'm fairly new to coding and currently im trying to build my first fat jar with gradle in eclipse. my project generates a html file with data i

I'm kinda confused about conversion of unicode characters into hexadecimal values. i'm using this website to get hexadecimal value for

Edit4: further review, it works if i send text in jp and translate it to en. i now believe that it's returning the string as utf-8 byte

I want to encode a string in android to utf-8. for example this string: grüne Ähren beißen flöhe to

Getting follwoing error while converting pandas dataframe to json overflowerror: unsupported utf-8 sequence length when

Ik building an go application where i want to output a cvs string from a buffer out via the http server. i'm putting it into the csv

I'm using a slot-scope to show text where there could be special characters like ñ or letters with accents, how to make sure those show up the

I'm trying to send buffers of data from a javascript application to a c++ application via zeromq (c++ and npm zeromq). if the buffer contains a

I am using nodejs as my backend, and sockets for a texting part of my application. i researched about using sockets and came across an interesting

I want to experiment with utf-8 and unicode, for that i want to build a small website which helps me to understand the encoding better.

Error occurs when opening docx file using python 3 when i tried to run:

My user input is a string, which also has emojis, so i have converted it to a list of utf8 code units. when i send this response to server, it

I need find if key contain dash, than need to get this key and value to new json file. this is my code: #coding=utf-8

Using scrapy crawler i am trying to extract data from html page and save the output as json file using command line: scrapy crawl

I have a bucket in s3 with a csv in it. there are no none-ascii characters in it. when i try to read it using python it will not let me.

I'm trying to convert all csv elements to python objects using following python script, but not all characters in csv file are in utf-8 and i've

In my following python code # coding: utf-8 # generate.py my_locale = [ { 'tag':'test_tag_long__body',

I'm trying to parse xml file in node with

I'm crawling some reading from the web and save them as utf8 txt:

In android when i am getting an utf-8 result from a server and i take the output of the server to a string what happens is i get extra

I am loading a folder path from a config.yml file. example name: c:/users/name/desktop/ü which contains an utf-8 character. when i

I am using the below code to encrypt and decrypt the data. now i want to encrypt the data from node js and want to decrypt the data from go lang.

Hello this is my first question on this site ever. i'm making a program that converts text to octal, but also octal to utf-8 text again.

I have downloaded my facebook data as json files. the json files for my posts contain emojis, which appear something like this in the json file:

I'm use curl -lk 'https://my-site.ru/?q=повар' and in nodejs server req.url i see

I have this error and i can't understand where is my mistake [error:flutter/lib/ui/ui_dart_state.cc(166)] unhandled exception:

I'm working with python 3.6.5 on a winows 10. i can not verify if a file exists in a directory or not. the problem seems to come from the

I'm currently building a website for a polish company and having some trouble with the special signs showing up as a square with demo in it: img

On one computer with the client (written in c) i get the error send: bad address when i try to send chars to another computer with a

I'm querying a database in iso-8859-1 but since node runs in utf8 mode, i must convert the data being returned this particular dbms. i

I have a react app embedded in wordpress page. it pulls content from a json api and displays it in various areas. my problem is that all

I am exporting a model in laravel 5.7, using the league/csv package:

I have this controller to redirect: $zone='لندن'; $type='خانه'; return redirect()->route('searchresult',['zone' =>

I have a file that has the mime-type: text/csv. i want to iterate through it and make a bunch of string-manipulation.

I have a codebase written in c++17 that makes heavy use of utf-8, and the

I have a problem to get data in my language with tis-620 [mysql structure] create table `foo` ( `id` int(11) not null,

Two questions in one, not sure if that's allowed, but they're directly related to the same code. i retrieve a csv string as a http response in

I'm having troubles with encoding text from android edittext when sending it with json to php script. first in android i get the text from

In my js i am trying to substring() text which generally works but unfortunately decapitates emojis. usatext =

I got an email with an attachment name that looks like: =?utf-8?b?cmvhbf9hdhrhy2htzw50x25hbwuuegxz?= however if

I would like to use python to convert utf8 special characters (accented, etc) to their extended ascii (purists are going to say there isn't such a

While saving an arabic word into mysql table, i am getting a string starting with &# and each block have semicolon. am using jsp as front

Here i have some simple code: #include <iostream> #include <cstdint> int main() { const unsigned char

I have a sqlite db and wanted to edit by using a text editor. so, i dumped the sqlite database file to plain text format and then edited and

I get a text from a database in utf-8 using json and it is in a file right now. when i print the data it is: توحید

I've a c program that looks like this: #include <stdio.h> #include <locale.h>

Based on mika tuupola's tutorial i am trying to use serial

Is there a way to convert cp437 to utf8 in browser using new

I am using laravel 5.2 and here are my database configuration settings: 'charset' => 'utf8', 'collation' =>

I know javascript’s regex only supports the \b word boundary class on ascii strings. but is there anything i can do to make it work with utf-8

I need to parse udp packets which can be invalid or contain some errors. i would like to replace invalid characters with . after a

I want to detect and/or replace weird utf, non-emoji characters that break my tokenization pipeline, like \uf0fc, which renders like

When using curl to take character streams off of the internet, when is the datastream converted from a multibyte datatype to a single byte

So the basic issue i am having is when i write to the txt file the unicode character \u2656 becomes this b'\xe2\x99\x96' (i

I insert korean characters in mysql database. but data shows nothing. as well as in phpmyadmin. anyone know what problem is and how to solve?

I want to encode the json responses of my api to utf-8, but every time i make a response i don't want to do this: return

I have the following program: #! /usr/bin/perl use strict; use warnings; use utf8; print

I have a csv with some data, and in one row there is a text that was added after encoding it in utf-8. this is the text:

I've got a weird case of a bom character (%ef%bb%bf) getting prepended to the output of the localization function __($key) in laravel 5.5

I am trying to fetch the records in laravel then it will give me following error. malformed utf-8 characters, possibly incorrectly

I have been going head over heels to solve this. i have a json feed that returns utf based data for example:

Git diff keeps showing non-utf8 characters. suggestion from

Is it safe to use ch >= '' && ch <=' ' as a condition that detects ascii whitespace. that is, i am

I have a laravel 5.6 installation, with config/database.php options for charset and collation set as utf8mb4 and

I was trying to do a query in my routes for the view composer to load all the records and count it and display it in my sidebar for purposes.

Ok so i simply call from mysql some text, and then i run it through nl2br to preserve linebreaks and all its symbols and what not. code

In python i could've converted it to unicode and do '(?u)^[w ]+$' regex search, but php doesn't seem to understand international w, or does

My table's collation is utf8_general_ci. my pages are encoded with utf-8 (without bom). within my pages, my equiv meta tag sets

I have a bunch of files with a mixtures of encodings mainly iso-8859-1 and utf-8. i would like to make all files utf-8, but when trying to

Is there any reason to prefer unicode(somestring, 'utf8') as opposed to somestring.decode('utf8')? my only

What's the best way to identify if a string (is or) might be utf-8 encoded? the win32 api istextunicode isn't of much help here.

I have an existing asp.net 2.0 website, stored in team foundation server 2005. some of the pages/controls are encoded as ansi (according to

I'm not sure if it's my system, although i haven't done anything unusual with it, but i've started noticing incorrectly rendered characters

For debugging purposes, i need to recursively search a directory for all files which start with a utf-8 byte order mark (bom). my current solution

I am writing a small app which i need to test with utf-8 characters of different number of byte lengths. i can input unicode characters to

One of the responses to a question i asked

Recently, someone asked about an algorithm for

I need to convert a bunch of files to utf-8 in python, and i have trouble with the "converting the file" part. i'd like to do the

I am looking for a method to compare and sort utf-8 strings in c++ in a case-insensitive manner to use it in a

Why is the following displayed different in linux vs windows? system.out.println(new string("¿".getbytes("utf-8"), "utf-8"));

How do i set the code page to utf-8 in a c windows program? i have a third party library that uses fopen to open files. i can use

I would like to know if there is an easy way to detect if the text on the clipboard is in iso 8859 or utf-8 ? here is my current code:

What i want to do is to remove all accents and umlauts from a string, turning "lärm" into "larm" or "andré" into "andre". what i tried to do was

I have a tapestry application that is serving its page as utf-8. that is, server responses have header: content-type:

I've been pulling my hear out over this problem for a few hours yesterday: i've a database on mysql 4.1.22 server with encoding set to

I'm using tinyxml to parse/build xml files. now, according to

I need to get utf-8 working in my java webapp (servlets + jsp, no framework used) to support äöå etc. for regular finnish text and

We have a set of applications that were developed for the ascii character set. now, we're trying to install it in iceland, and are running into

I currentyl have no clue on how to sort an array which contains utf-8 encoded strings in php. the array comes from a ldap server so sorting via a

How do i truncate a java string so that i know it will fit in a given number of bytes storage once it is utf-8 encoded?

I'm processing some data files that are supposed to be valid utf-8 but aren't, which causes the parser (not under my control) to fail. i'd like to

Is there a sql or php script that i can run that will change the default collation in all tables and fields in a

I'm still trying to decide whether my (home) project should use utf-8

Trying the easy approach: sqlite2 mydb.db .dump | sqlite3 mydb-new.db i got this error: